2024-03-19T00:53:24.850 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-19T00:53:24.850 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} --verbose --archive /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283 --name pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/orig.config.yaml 2024-03-19T00:53:24.902 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-19T00:53:24.988 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283 branch: wip-batrick-testing-20240318.181317 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7609283' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240318.181317 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - PG_AVAILABILITY - PG_DEGRADED - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240318.181317 sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3005 sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 sleep_before_teardown: 0 subset: 433/512 suite: fs suite_branch: wip-batrick-testing-20240318.181317 suite_path: /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: e48b1f6c0608b2615e1890cc46286c3877d6e910 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDodgWjm5KBLAUCgPEqL6PcfPJyqSgTP63zjcD/2kOYd58LAmNW5zN0DW4oxUxJ0vGTakCrym7Oza9JneQ8Qas4= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPcgf3bF3xOYK0jSmjAbx92BWc8p/GwxO36T7l69Ur4v9h1TBC3+I3HAUT7Z1FTLwwl6VZM7dkTMl0XLGOuUM3A= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFytUTgr0V6wZPcYc/2NVkHEvKQSgGFiplTT0K+kTA9nc0hnOjnedq4YECj44Zu3+yZ4rqw5sJdsu32/5StP8TA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-18_21:05:49 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-19T00:53:24.989 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa; will attempt to use it 2024-03-19T00:53:24.989 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_e48b1f6c0608b2615e1890cc46286c3877d6e910/qa/tasks 2024-03-19T00:53:24.989 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-19T00:53:24.991 INFO:teuthology.task.internal:Checking packages... 2024-03-19T00:53:25.018 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b6ae24918b03b1b8a19c9263a50655bba80bb810' 2024-03-19T00:53:25.018 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-19T00:53:25.018 INFO:teuthology.packaging:ref: None 2024-03-19T00:53:25.018 INFO:teuthology.packaging:tag: None 2024-03-19T00:53:25.018 INFO:teuthology.packaging:branch: wip-batrick-testing-20240318.181317 2024-03-19T00:53:25.018 INFO:teuthology.packaging:sha1: b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T00:53:25.019 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240318.181317 2024-03-19T00:53:25.166 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2206.gb6ae2491 2024-03-19T00:53:25.168 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-19T00:53:25.175 INFO:teuthology.task.internal:no buildpackages task found 2024-03-19T00:53:25.175 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-19T00:53:25.214 INFO:teuthology.task.internal:Saving configuration 2024-03-19T00:53:25.237 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-19T00:53:25.245 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T00:53:25.270 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:48:25.049898', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDodgWjm5KBLAUCgPEqL6PcfPJyqSgTP63zjcD/2kOYd58LAmNW5zN0DW4oxUxJ0vGTakCrym7Oza9JneQ8Qas4='} 2024-03-19T00:53:25.295 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:48:25.051219', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPcgf3bF3xOYK0jSmjAbx92BWc8p/GwxO36T7l69Ur4v9h1TBC3+I3HAUT7Z1FTLwwl6VZM7dkTMl0XLGOuUM3A='} 2024-03-19T00:53:25.315 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:48:25.048381', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFytUTgr0V6wZPcYc/2NVkHEvKQSgGFiplTT0K+kTA9nc0hnOjnedq4YECj44Zu3+yZ4rqw5sJdsu32/5StP8TA='} 2024-03-19T00:53:25.316 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-19T00:53:25.339 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-19T00:53:25.339 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-19T00:53:25.339 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-19T00:53:25.339 INFO:teuthology.run_tasks:Running task console_log... 2024-03-19T00:53:25.454 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f96835b2160>, signals=[15]) 2024-03-19T00:53:25.455 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-19T00:53:25.461 INFO:teuthology.task.internal:Opening connections... 2024-03-19T00:53:25.461 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-03-19T00:53:25.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:53:25.554 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-03-19T00:53:25.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:53:25.644 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-03-19T00:53:25.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T00:53:25.735 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-19T00:53:25.742 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-03-19T00:53:25.760 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-03-19T00:53:25.761 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-03-19T00:53:25.817 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-03-19T00:53:25.817 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-03-19T00:53:25.817 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-03-19T00:53:25.817 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-03-19T00:53:25.817 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-03-19T00:53:25.818 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-03-19T00:53:25.818 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-19T00:53:25.818 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-03-19T00:53:25.818 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-03-19T00:53:25.818 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-19T00:53:25.818 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-03-19T00:53:25.818 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-19T00:53:25.818 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-19T00:53:25.818 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-19T00:53:25.819 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-03-19T00:53:25.843 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-03-19T00:53:25.861 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-03-19T00:53:25.861 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-03-19T00:53:25.918 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-03-19T00:53:25.918 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-03-19T00:53:25.918 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-03-19T00:53:25.918 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-03-19T00:53:25.918 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-03-19T00:53:25.918 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-03-19T00:53:25.918 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-19T00:53:25.918 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-03-19T00:53:25.918 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-03-19T00:53:25.919 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-19T00:53:25.919 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-03-19T00:53:25.919 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-19T00:53:25.919 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-19T00:53:25.919 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-19T00:53:25.919 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-03-19T00:53:25.943 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-03-19T00:53:25.961 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-03-19T00:53:25.961 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-03-19T00:53:26.018 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-03-19T00:53:26.018 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-03-19T00:53:26.018 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-19T00:53:26.019 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-19T00:53:26.020 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-03-19T00:53:26.042 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-19T00:53:26.052 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-19T00:53:26.071 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-19T00:53:26.071 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-03-19T00:53:26.074 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-03-19T00:53:26.076 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-03-19T00:53:26.093 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-19T00:53:26.099 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-19T00:53:26.099 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-03-19T00:53:26.130 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-03-19T00:53:26.133 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-03-19T00:53:26.243 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-19T00:53:26.314 INFO:teuthology.run_tasks:Running task kernel... 2024-03-19T00:53:26.329 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-19T00:53:26.329 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-19T00:53:26.329 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-19T00:53:26.329 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-19T00:53:26.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T00:53:26.346 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-03-19T00:53:26.402 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-361.el9.x86_64 2024-03-19T00:53:26.403 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-361.el9.x86_64 2024-03-19T00:53:26.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-03-19T00:53:28.827 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - AppStream 8.1 MB/s | 6.7 MB 00:00 2024-03-19T00:53:30.244 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2024-03-19T00:53:31.101 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-19T00:53:35.119 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 19 MB 00:02 2024-03-19T01:06:43.878 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 8.0 kB/s | 6.1 MB 13:05 2024-03-19T01:06:45.694 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-03-19T01:07:17.243 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 673 kB/s | 21 MB 00:31 2024-03-19T01:07:22.106 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-19T01:07:24.195 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-19T01:07:24.195 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-19T01:07:24.287 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-03-19T01:07:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-03-19T01:07:24.291 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-03-19T01:07:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-03-19T01:07:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-03-19T01:07:24.291 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-19T01:07:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-03-19T01:07:24.291 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-19T01:07:24.291 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-19T01:07:24.291 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-19T01:07:24.292 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:07:24.292 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-03-19T01:07:24.292 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-03-19T01:07:24.292 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-03-19T01:07:24.292 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:07:24.292 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 76 M 2024-03-19T01:07:24.292 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 123 M 2024-03-19T01:07:24.292 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-03-19T01:07:24.790 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 126 kB/s | 44 kB 00:00 2024-03-19T01:07:38.088 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 1.1 MB/s | 15 MB 00:13 2024-03-19T01:07:47.719 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 1.4 MB/s | 33 MB 00:23 2024-03-19T01:07:56.519 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): kernel-modules-core-5.14.0-430.el9.x86_6 886 kB/s | 27 MB 00:31 2024-03-19T01:07:56.519 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-03-19T01:07:56.519 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.4 MB/s | 76 MB 00:32 2024-03-19T01:07:57.186 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-03-19T01:07:57.551 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-03-19T01:07:57.551 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-03-19T01:07:58.546 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-03-19T01:07:58.546 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-03-19T01:08:01.025 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-03-19T01:08:01.656 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-19T01:08:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-19T01:08:04.466 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-19T01:08:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-19T01:08:09.273 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-19T01:08:09.363 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-19T01:08:14.072 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-19T01:08:43.602 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-19T01:08:43.603 INFO:teuthology.orchestra.run.smithi092.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-19T01:08:43.603 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:08:43.650 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-19T01:08:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-19T01:08:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-19T01:08:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-19T01:08:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-19T01:08:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-19T01:08:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:08:44.470 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-03-19T01:08:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-19T01:08:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-19T01:08:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-19T01:08:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-19T01:08:44.470 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:08:44.470 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-03-19T01:08:44.809 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-03-19T01:08:44.947 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-19T01:08:44.947 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-19T01:08:44.947 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-19T01:08:44.947 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-19T01:08:44.947 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-19T01:08:44.947 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-19T01:08:44.947 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-03-19T01:08:44.948 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-03-19T01:08:45.027 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-19T01:08:45.032 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-19T01:08:45.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2024-03-19T01:08:45.070 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2024-03-19T01:08:45.070 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2024-03-19T01:08:45.070 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2024-03-19T01:08:45.070 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 68.el9 2024-03-19T01:08:45.070 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2024-03-19T01:08:45.070 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8264254 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-19T01:08:45.071 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-19T01:08:45.072 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-19T01:08:45.072 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2024-03-19T01:08:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:08:45.072 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2024-03-19T01:08:45.073 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-19T01:08:45.073 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 5.14.0-430.el9.x86_64 2024-03-19T01:08:45.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-19T01:08:45.652 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-03-19T01:08:47.066 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-19T01:08:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-03-19T01:08:47.107 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-03-19T01:08:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.jf74NEYSIu 2024-03-19T01:08:47.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jf74NEYSIu 2024-03-19T01:08:47.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.jf74NEYSIu 2024-03-19T01:08:47.333 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.jf74NEYSIu is 6KB 2024-03-19T01:08:47.382 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.jf74NEYSIu 2024-03-19T01:08:47.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2024-03-19T01:08:47.461 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-19T01:08:47.461 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-19T01:08:47.461 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-19T01:08:47.461 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-19T01:08:47.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-19T01:08:47.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-03-19T01:09:17.611 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-19T01:09:17.611 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-19T01:09:17.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T01:09:36.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-19T01:09:45.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-19T01:09:45.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T01:09:48.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-19T01:10:00.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-19T01:10:00.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T01:10:18.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-19T01:10:33.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-19T01:10:33.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T01:10:33.957 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-03-19T01:10:34.317 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-19T01:10:34.317 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-430.el9.x86_64"... 2024-03-19T01:10:34.317 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-03-19T01:10:34.375 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-430.el9.x86_64 2024-03-19T01:10:34.376 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-19T01:10:34.376 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-19T01:10:34.376 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-19T01:10:35.377 DEBUG:teuthology.parallel:result is None 2024-03-19T01:10:35.377 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-19T01:10:35.385 INFO:teuthology.task.internal:Creating test directory... 2024-03-19T01:10:35.385 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-19T01:10:35.388 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-19T01:10:35.390 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-19T01:10:35.409 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-19T01:10:35.423 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-19T01:10:35.472 INFO:teuthology.task.internal:Creating archive directory... 2024-03-19T01:10:35.473 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-19T01:10:35.475 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-19T01:10:35.477 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-19T01:10:35.602 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-19T01:10:35.609 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-19T01:10:35.609 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-19T01:10:35.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:10:35.626 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-19T01:10:35.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:10:35.642 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-19T01:10:35.644 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-19T01:10:35.668 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-19T01:10:35.704 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T01:10:35.714 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T01:10:35.721 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T01:10:35.743 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T01:10:35.764 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T01:10:35.768 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-19T01:10:35.770 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-19T01:10:35.778 INFO:teuthology.task.internal:Configuring sudo... 2024-03-19T01:10:35.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-19T01:10:35.780 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-19T01:10:35.782 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-19T01:10:35.813 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-19T01:10:35.822 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-19T01:10:35.822 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-19T01:10:35.858 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-19T01:10:35.861 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-19T01:10:35.876 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T01:10:35.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T01:10:36.030 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T01:10:36.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T01:10:36.117 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:10:36.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-19T01:10:36.187 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T01:10:36.228 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T01:10:36.314 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T01:10:36.357 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T01:10:36.402 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:10:36.402 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-19T01:10:36.472 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T01:10:36.516 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-19T01:10:36.569 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T01:10:36.651 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-19T01:10:36.696 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:10:36.696 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-19T01:10:36.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-03-19T01:10:36.767 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-03-19T01:10:36.770 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-03-19T01:10:36.828 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T01:10:36.842 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T01:10:36.853 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T01:10:37.402 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-19T01:10:37.410 INFO:teuthology.task.internal:Starting timer... 2024-03-19T01:10:37.410 INFO:teuthology.run_tasks:Running task pcp... 2024-03-19T01:10:37.460 INFO:teuthology.run_tasks:Running task selinux... 2024-03-19T01:10:37.509 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-19T01:10:37.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-03-19T01:10:37.556 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-03-19T01:10:37.557 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-03-19T01:10:37.646 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-03-19T01:10:37.648 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-03-19T01:10:37.718 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-03-19T01:10:37.720 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-19T01:10:37.720 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-03-19T01:10:37.739 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-03-19T01:10:37.740 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-03-19T01:10:37.773 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-03-19T01:10:37.773 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-03-19T01:10:37.798 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-03-19T01:10:37.798 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive'} 2024-03-19T01:10:37.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T01:10:37.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:10:37.825 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T01:10:37.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:10:37.855 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T01:10:37.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:10:37.882 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-19T01:10:37.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-03-19T01:10:37.950 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-03-19T01:10:37.983 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-03-19T01:10:38.016 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-19T01:10:38.026 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-19T01:10:38.031 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-19T01:10:38.197 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-19T01:10:38.214 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-19T01:10:38.216 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi144.front.sepia.ceph.com,smithi159.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-19T01:24:12.791 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com')] 2024-03-19T01:24:12.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-19T01:24:12.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T01:24:12.924 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-03-19T01:24:13.001 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-19T01:24:13.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-19T01:24:13.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T01:24:13.094 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-03-19T01:24:13.161 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-19T01:24:13.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-19T01:24:13.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-19T01:24:13.253 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-03-19T01:24:13.331 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-19T01:24:13.331 INFO:teuthology.run_tasks:Running task clock... 2024-03-19T01:24:13.343 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-19T01:24:13.343 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-19T01:24:13.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T01:24:13.346 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-19T01:24:13.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T01:24:13.349 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-19T01:24:13.349 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T01:24:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-19T01:24:13.379 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-19T01:24:13.392 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-19T01:24:13.396 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-19T01:24:13.404 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-19T01:24:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-03-19T01:24:13.422 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-19T01:24:13.426 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-03-19T01:24:13.435 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-03-19T01:24:13.440 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-03-19T01:24:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-03-19T01:24:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-19T01:24:13.456 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-19T01:24:13.465 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-03-19T01:24:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-19T01:24:13.473 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-19T01:24:13.483 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-19T01:24:13.499 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-19T01:24:13.516 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-19T01:24:13.517 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-03-19T01:24:13.520 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T01:24:13.520 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-19T01:24:13.520 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.520 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.520 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.520 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.521 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T01:24:13.521 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-03-19T01:24:13.521 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.521 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.521 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.521 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.543 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-03-19T01:24:13.547 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T01:24:13.547 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-03-19T01:24:13.547 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.547 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.548 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.548 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T01:24:13.548 INFO:teuthology.run_tasks:Running task install... 2024-03-19T01:24:13.557 DEBUG:teuthology.task.install:project ceph 2024-03-19T01:24:13.557 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'}} 2024-03-19T01:24:13.557 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810'} 2024-03-19T01:24:13.557 INFO:teuthology.task.install:Using flavor: default 2024-03-19T01:24:13.564 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-19T01:24:13.565 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-19T01:24:13.565 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-19T01:24:13.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:24:13.567 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-19T01:24:13.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:24:13.568 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'tag': None, 'wait_for_package': False} 2024-03-19T01:24:13.568 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:24:13.715 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-19T01:24:13.715 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-19T01:24:13.792 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-19T01:24:13.792 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-19T01:24:13.801 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/ 2024-03-19T01:24:13.801 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2206.gb6ae2491 2024-03-19T01:24:13.842 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-19T01:24:13.842 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:24:13.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-19T01:24:13.872 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-19T01:24:13.872 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-19T01:24:13.911 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-19T01:24:13.911 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:24:13.911 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-19T01:24:13.940 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-19T01:24:13.940 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-19T01:24:13.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-19T01:24:13.964 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-19T01:24:13.965 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:24:13.965 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-19T01:24:13.993 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-19T01:24:13.994 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b6ae24918b03b1b8a19c9263a50655bba80bb810/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-19T01:24:14.009 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-19T01:24:14.021 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-19T01:24:14.062 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-19T01:24:14.088 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-03-19T01:24:14.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-03-19T01:24:14.090 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-19T01:24:14.143 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-19T01:24:14.157 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2024-03-19T01:24:14.158 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2024-03-19T01:24:14.209 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-03-19T01:24:14.211 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-03-19T01:24:14.327 INFO:teuthology.orchestra.run.smithi092.stdout:74 files removed 2024-03-19T01:24:14.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-19T01:24:14.394 INFO:teuthology.orchestra.run.smithi159.stdout:74 files removed 2024-03-19T01:24:14.420 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-19T01:24:14.446 INFO:teuthology.orchestra.run.smithi144.stdout:74 files removed 2024-03-19T01:24:14.473 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-19T01:24:15.123 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-03-19T01:24:15.128 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-03-19T01:24:15.207 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 188 kB/s | 83 kB 00:00 2024-03-19T01:24:15.448 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 35 kB/s | 10 kB 00:00 2024-03-19T01:24:15.662 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 21 kB/s | 10 kB 00:00 2024-03-19T01:24:15.699 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 22 kB/s | 10 kB 00:00 2024-03-19T01:24:15.849 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-03-19T01:24:16.038 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-19T01:24:16.083 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-03-19T01:24:16.369 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-19T01:24:16.490 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-19T01:24:16.574 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-19T01:24:17.944 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-03-19T01:24:17.953 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-03-19T01:24:18.132 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-19T01:24:20.194 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-03-19T01:24:20.431 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.0 MB 00:01 2024-03-19T01:24:31.518 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 1.8 MB/s | 19 MB 00:10 2024-03-19T01:24:36.435 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.1 MB 00:01 2024-03-19T01:24:38.195 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-03-19T01:24:39.797 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-19T01:24:44.782 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-19T01:24:47.120 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T01:24:47.121 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T01:24:47.135 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-19T01:24:47.136 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-19T01:24:47.137 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-19T01:24:47.138 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-19T01:24:47.140 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-19T01:24:47.143 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-19T01:24:47.144 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-19T01:24:47.225 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-19T01:24:47.237 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-19T01:24:47.237 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-19T01:24:47.238 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-19T01:24:47.239 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-19T01:24:47.240 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-19T01:24:47.241 INFO:teuthology.orchestra.run.smithi144.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-19T01:24:47.242 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-19T01:24:47.243 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-19T01:24:47.244 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-19T01:24:47.245 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-19T01:24:47.246 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-19T01:24:47.247 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-19T01:24:47.248 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-19T01:24:47.249 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-19T01:24:47.250 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout:Install 150 Packages 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 24 Packages 2024-03-19T01:24:47.251 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:24:47.252 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 224 M 2024-03-19T01:24:47.252 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-19T01:24:48.092 INFO:teuthology.orchestra.run.smithi144.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-03-19T01:24:48.334 INFO:teuthology.orchestra.run.smithi144.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 3.6 MB/s | 891 kB 00:00 2024-03-19T01:24:48.409 INFO:teuthology.orchestra.run.smithi144.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 1.9 MB/s | 145 kB 00:00 2024-03-19T01:24:48.842 INFO:teuthology.orchestra.run.smithi144.stdout:(4/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 5.4 MB/s | 2.4 MB 00:00 2024-03-19T01:24:48.951 INFO:teuthology.orchestra.run.smithi144.stdout:(5/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 5.2 MB/s | 5.4 MB 00:01 2024-03-19T01:24:49.176 INFO:teuthology.orchestra.run.smithi144.stdout:(6/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-03-19T01:24:49.510 INFO:teuthology.orchestra.run.smithi144.stdout:(7/174): ceph-common-19.0.0-2206.gb6ae2491.el9. 13 MB/s | 21 MB 00:01 2024-03-19T01:24:49.785 INFO:teuthology.orchestra.run.smithi144.stdout:(8/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 5.6 MB/s | 4.7 MB 00:00 2024-03-19T01:24:49.960 INFO:teuthology.orchestra.run.smithi144.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 144 kB/s | 25 kB 00:00 2024-03-19T01:24:50.127 INFO:teuthology.orchestra.run.smithi144.stdout:(10/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el 16 MB/s | 9.8 MB 00:00 2024-03-19T01:24:50.194 INFO:teuthology.orchestra.run.smithi144.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 3.3 MB/s | 221 kB 00:00 2024-03-19T01:24:50.244 INFO:teuthology.orchestra.run.smithi144.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 543 kB/s | 27 kB 00:00 2024-03-19T01:24:50.286 INFO:teuthology.orchestra.run.smithi144.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 766 kB/s | 32 kB 00:00 2024-03-19T01:24:50.387 INFO:teuthology.orchestra.run.smithi144.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 7.3 MB/s | 746 kB 00:00 2024-03-19T01:24:50.437 INFO:teuthology.orchestra.run.smithi144.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 3.2 MB/s | 163 kB 00:00 2024-03-19T01:24:50.495 INFO:teuthology.orchestra.run.smithi144.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 2.1 MB/s | 127 kB 00:00 2024-03-19T01:24:50.570 INFO:teuthology.orchestra.run.smithi144.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 6.5 MB/s | 501 kB 00:00 2024-03-19T01:24:50.946 INFO:teuthology.orchestra.run.smithi144.stdout:(18/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 14 MB/s | 5.1 MB 00:00 2024-03-19T01:24:50.996 INFO:teuthology.orchestra.run.smithi144.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 908 kB/s | 45 kB 00:00 2024-03-19T01:24:51.063 INFO:teuthology.orchestra.run.smithi144.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 1.9 MB/s | 131 kB 00:00 2024-03-19T01:24:51.130 INFO:teuthology.orchestra.run.smithi144.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 2.4 MB/s | 162 kB 00:00 2024-03-19T01:24:51.206 INFO:teuthology.orchestra.run.smithi144.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 4.2 MB/s | 321 kB 00:00 2024-03-19T01:24:51.281 INFO:teuthology.orchestra.run.smithi144.stdout:(23/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 3.9 MB/s | 298 kB 00:00 2024-03-19T01:24:51.348 INFO:teuthology.orchestra.run.smithi144.stdout:(24/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.5 MB/s | 99 kB 00:00 2024-03-19T01:24:51.515 INFO:teuthology.orchestra.run.smithi144.stdout:(25/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 7.2 MB/s | 17 MB 00:02 2024-03-19T01:24:51.540 INFO:teuthology.orchestra.run.smithi144.stdout:(26/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 445 kB/s | 85 kB 00:00 2024-03-19T01:24:51.599 INFO:teuthology.orchestra.run.smithi144.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-19T01:24:51.658 INFO:teuthology.orchestra.run.smithi144.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 583 kB/s | 34 kB 00:00 2024-03-19T01:24:51.716 INFO:teuthology.orchestra.run.smithi144.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 2.3 MB/s | 139 kB 00:00 2024-03-19T01:24:51.875 INFO:teuthology.orchestra.run.smithi144.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 11 MB/s | 1.8 MB 00:00 2024-03-19T01:24:51.933 INFO:teuthology.orchestra.run.smithi144.stdout:(31/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 7.4 MB/s | 3.1 MB 00:00 2024-03-19T01:24:52.025 INFO:teuthology.orchestra.run.smithi144.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2206.gb6 2.7 MB/s | 248 kB 00:00 2024-03-19T01:24:52.125 INFO:teuthology.orchestra.run.smithi144.stdout:(33/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 494 kB/s | 49 kB 00:00 2024-03-19T01:24:52.209 INFO:teuthology.orchestra.run.smithi144.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2206.gb 198 kB/s | 16 kB 00:00 2024-03-19T01:24:52.310 INFO:teuthology.orchestra.run.smithi144.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-03-19T01:24:52.335 INFO:teuthology.orchestra.run.smithi144.stdout:(36/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 2.1 MB/s | 264 kB 00:00 2024-03-19T01:24:52.393 INFO:teuthology.orchestra.run.smithi144.stdout:(37/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 8.8 MB/s | 749 kB 00:00 2024-03-19T01:24:52.419 INFO:teuthology.orchestra.run.smithi144.stdout:(38/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 218 kB/s | 18 kB 00:00 2024-03-19T01:24:52.569 INFO:teuthology.orchestra.run.smithi144.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 662 kB 00:00 2024-03-19T01:24:52.602 INFO:teuthology.orchestra.run.smithi144.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.9 MB/s | 920 kB 00:00 2024-03-19T01:24:52.628 INFO:teuthology.orchestra.run.smithi144.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-19T01:24:52.653 INFO:teuthology.orchestra.run.smithi144.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-19T01:24:52.670 INFO:teuthology.orchestra.run.smithi144.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 767 kB/s | 32 kB 00:00 2024-03-19T01:24:52.754 INFO:teuthology.orchestra.run.smithi144.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-19T01:24:52.770 INFO:teuthology.orchestra.run.smithi144.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 172 kB/s | 17 kB 00:00 2024-03-19T01:24:52.796 INFO:teuthology.orchestra.run.smithi144.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 407 kB/s | 10 kB 00:00 2024-03-19T01:24:52.821 INFO:teuthology.orchestra.run.smithi144.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-19T01:24:52.846 INFO:teuthology.orchestra.run.smithi144.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-19T01:24:52.880 INFO:teuthology.orchestra.run.smithi144.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-19T01:24:52.905 INFO:teuthology.orchestra.run.smithi144.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-03-19T01:24:52.931 INFO:teuthology.orchestra.run.smithi144.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 938 kB/s | 46 kB 00:00 2024-03-19T01:24:52.956 INFO:teuthology.orchestra.run.smithi144.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-19T01:24:52.981 INFO:teuthology.orchestra.run.smithi144.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-19T01:24:53.007 INFO:teuthology.orchestra.run.smithi144.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-19T01:24:53.032 INFO:teuthology.orchestra.run.smithi144.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-03-19T01:24:53.132 INFO:teuthology.orchestra.run.smithi144.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-19T01:24:53.174 INFO:teuthology.orchestra.run.smithi144.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-19T01:24:53.207 INFO:teuthology.orchestra.run.smithi144.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-19T01:24:53.233 INFO:teuthology.orchestra.run.smithi144.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-19T01:24:53.317 INFO:teuthology.orchestra.run.smithi144.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-03-19T01:24:53.384 INFO:teuthology.orchestra.run.smithi144.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 32 MB/s | 5.7 MB 00:00 2024-03-19T01:24:53.408 INFO:teuthology.orchestra.run.smithi144.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 860 kB/s | 78 kB 00:00 2024-03-19T01:24:53.526 INFO:teuthology.orchestra.run.smithi144.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2024-03-19T01:24:53.551 INFO:teuthology.orchestra.run.smithi144.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-19T01:24:53.576 INFO:teuthology.orchestra.run.smithi144.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-03-19T01:24:53.602 INFO:teuthology.orchestra.run.smithi144.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-19T01:24:53.627 INFO:teuthology.orchestra.run.smithi144.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-19T01:24:53.719 INFO:teuthology.orchestra.run.smithi144.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 4.8 MB 00:00 2024-03-19T01:24:53.744 INFO:teuthology.orchestra.run.smithi144.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 217 kB 00:00 2024-03-19T01:24:53.769 INFO:teuthology.orchestra.run.smithi144.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-19T01:24:53.795 INFO:teuthology.orchestra.run.smithi144.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-19T01:24:53.820 INFO:teuthology.orchestra.run.smithi144.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-03-19T01:24:53.845 INFO:teuthology.orchestra.run.smithi144.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-19T01:24:53.870 INFO:teuthology.orchestra.run.smithi144.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-19T01:24:53.896 INFO:teuthology.orchestra.run.smithi144.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-19T01:24:53.921 INFO:teuthology.orchestra.run.smithi144.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 947 kB/s | 47 kB 00:00 2024-03-19T01:24:53.946 INFO:teuthology.orchestra.run.smithi144.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 855 kB/s | 43 kB 00:00 2024-03-19T01:24:54.122 INFO:teuthology.orchestra.run.smithi144.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 224 kB/s | 39 kB 00:00 2024-03-19T01:24:54.272 INFO:teuthology.orchestra.run.smithi144.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-03-19T01:24:54.306 INFO:teuthology.orchestra.run.smithi144.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 225 kB/s | 41 kB 00:00 2024-03-19T01:24:54.331 INFO:teuthology.orchestra.run.smithi144.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-19T01:24:54.356 INFO:teuthology.orchestra.run.smithi144.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-19T01:24:54.382 INFO:teuthology.orchestra.run.smithi144.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-19T01:24:54.407 INFO:teuthology.orchestra.run.smithi144.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 875 kB/s | 44 kB 00:00 2024-03-19T01:24:54.432 INFO:teuthology.orchestra.run.smithi144.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-19T01:24:54.457 INFO:teuthology.orchestra.run.smithi144.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-19T01:24:54.891 INFO:teuthology.orchestra.run.smithi144.stdout:(87/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 9.7 MB/s | 48 MB 00:04 2024-03-19T01:24:55.008 INFO:teuthology.orchestra.run.smithi144.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.1 MB 00:00 2024-03-19T01:24:55.033 INFO:teuthology.orchestra.run.smithi144.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 179 kB/s | 103 kB 00:00 2024-03-19T01:24:55.059 INFO:teuthology.orchestra.run.smithi144.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 752 kB/s | 124 kB 00:00 2024-03-19T01:24:55.084 INFO:teuthology.orchestra.run.smithi144.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-19T01:27:03.895 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] jq-1.6-16.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/jq-1.6-16.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T01:27:03.895 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T01:27:03.896 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T01:27:04.514 INFO:teuthology.orchestra.run.smithi144.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.4 kB/s | 187 kB 02:09 2024-03-19T01:27:04.539 INFO:teuthology.orchestra.run.smithi144.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 kB/s | 195 kB 02:09 2024-03-19T01:27:04.714 INFO:teuthology.orchestra.run.smithi144.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 624 kB/s | 125 kB 00:00 2024-03-19T01:27:04.806 INFO:teuthology.orchestra.run.smithi144.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 80 kB/s | 21 kB 00:00 2024-03-19T01:27:04.990 INFO:teuthology.orchestra.run.smithi144.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 92 kB/s | 25 kB 00:00 2024-03-19T01:27:05.040 INFO:teuthology.orchestra.run.smithi144.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 6.2 kB/s | 803 kB 02:09 2024-03-19T01:27:05.357 INFO:teuthology.orchestra.run.smithi144.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 669 kB/s | 245 kB 00:00 2024-03-19T01:27:05.549 INFO:teuthology.orchestra.run.smithi144.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.1 MB/s | 216 kB 00:00 2024-03-19T01:27:05.574 INFO:teuthology.orchestra.run.smithi144.stdout:(100/174): python3-jinja2-2.11.3-5.el9.noarch.r 464 kB/s | 248 kB 00:00 2024-03-19T01:27:05.649 INFO:teuthology.orchestra.run.smithi144.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 347 kB/s | 35 kB 00:00 2024-03-19T01:27:05.675 INFO:teuthology.orchestra.run.smithi144.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 443 kB/s | 44 kB 00:00 2024-03-19T01:27:05.750 INFO:teuthology.orchestra.run.smithi144.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 936 kB/s | 94 kB 00:00 2024-03-19T01:27:06.625 INFO:teuthology.orchestra.run.smithi144.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.3 MB/s | 4.1 MB 00:01 2024-03-19T01:27:06.809 INFO:teuthology.orchestra.run.smithi144.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.9 MB/s | 548 kB 00:00 2024-03-19T01:27:06.876 INFO:teuthology.orchestra.run.smithi144.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-03-19T01:27:06.934 INFO:teuthology.orchestra.run.smithi144.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-03-19T01:27:07.634 INFO:teuthology.orchestra.run.smithi144.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-03-19T01:27:07.685 INFO:teuthology.orchestra.run.smithi144.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-03-19T01:27:07.735 INFO:teuthology.orchestra.run.smithi144.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-03-19T01:27:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-03-19T01:27:07.810 INFO:teuthology.orchestra.run.smithi144.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-03-19T01:27:07.902 INFO:teuthology.orchestra.run.smithi144.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-03-19T01:27:07.969 INFO:teuthology.orchestra.run.smithi144.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-03-19T01:27:08.011 INFO:teuthology.orchestra.run.smithi144.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-03-19T01:27:08.053 INFO:teuthology.orchestra.run.smithi144.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 773 kB/s | 32 kB 00:00 2024-03-19T01:27:08.095 INFO:teuthology.orchestra.run.smithi144.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 358 kB/s | 15 kB 00:00 2024-03-19T01:27:08.137 INFO:teuthology.orchestra.run.smithi144.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-03-19T01:27:08.196 INFO:teuthology.orchestra.run.smithi144.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-03-19T01:27:08.246 INFO:teuthology.orchestra.run.smithi144.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-03-19T01:27:08.288 INFO:teuthology.orchestra.run.smithi144.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 850 kB/s | 35 kB 00:00 2024-03-19T01:27:08.330 INFO:teuthology.orchestra.run.smithi144.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 4.7 MB/s | 203 kB 00:00 2024-03-19T01:27:08.480 INFO:teuthology.orchestra.run.smithi144.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-03-19T01:27:08.522 INFO:teuthology.orchestra.run.smithi144.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-03-19T01:27:08.564 INFO:teuthology.orchestra.run.smithi144.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-03-19T01:27:08.597 INFO:teuthology.orchestra.run.smithi144.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 530 kB/s | 18 kB 00:00 2024-03-19T01:27:08.631 INFO:teuthology.orchestra.run.smithi144.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 697 kB/s | 23 kB 00:00 2024-03-19T01:27:08.665 INFO:teuthology.orchestra.run.smithi144.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 582 kB/s | 19 kB 00:00 2024-03-19T01:27:08.698 INFO:teuthology.orchestra.run.smithi144.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 593 kB/s | 20 kB 00:00 2024-03-19T01:27:08.790 INFO:teuthology.orchestra.run.smithi144.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-03-19T01:27:08.832 INFO:teuthology.orchestra.run.smithi144.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-03-19T01:27:08.874 INFO:teuthology.orchestra.run.smithi144.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-03-19T01:27:08.916 INFO:teuthology.orchestra.run.smithi144.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-03-19T01:27:08.958 INFO:teuthology.orchestra.run.smithi144.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-03-19T01:27:09.000 INFO:teuthology.orchestra.run.smithi144.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 396 kB/s | 16 kB 00:00 2024-03-19T01:27:09.042 INFO:teuthology.orchestra.run.smithi144.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-03-19T01:27:09.084 INFO:teuthology.orchestra.run.smithi144.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 678 kB/s | 28 kB 00:00 2024-03-19T01:27:09.126 INFO:teuthology.orchestra.run.smithi144.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 740 kB/s | 31 kB 00:00 2024-03-19T01:27:09.168 INFO:teuthology.orchestra.run.smithi144.stdout:(139/174): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-03-19T01:27:09.209 INFO:teuthology.orchestra.run.smithi144.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-03-19T01:27:09.251 INFO:teuthology.orchestra.run.smithi144.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 857 kB/s | 36 kB 00:00 2024-03-19T01:27:09.293 INFO:teuthology.orchestra.run.smithi144.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-03-19T01:27:09.335 INFO:teuthology.orchestra.run.smithi144.stdout:(143/174): python3-webob-1.8.7-6.el9.noarch.rpm 5.4 MB/s | 230 kB 00:00 2024-03-19T01:27:09.377 INFO:teuthology.orchestra.run.smithi144.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-03-19T01:27:09.427 INFO:teuthology.orchestra.run.smithi144.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-03-19T01:27:09.469 INFO:teuthology.orchestra.run.smithi144.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 481 kB/s | 20 kB 00:00 2024-03-19T01:27:09.511 INFO:teuthology.orchestra.run.smithi144.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-03-19T01:27:09.620 INFO:teuthology.orchestra.run.smithi144.stdout:(148/174): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-03-19T01:27:10.270 INFO:teuthology.orchestra.run.smithi144.stdout:(149/174): librados2-19.0.0-2206.gb6ae2491.el9. 5.2 MB/s | 3.4 MB 00:00 2024-03-19T01:27:10.633 INFO:teuthology.orchestra.run.smithi144.stdout:(150/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 8.7 MB/s | 3.1 MB 00:00 2024-03-19T01:27:10.912 INFO:teuthology.orchestra.run.smithi144.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.1 MB/s | 321 kB 00:00 2024-03-19T01:27:13.629 INFO:teuthology.orchestra.run.smithi144.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 869 kB/s | 2.3 MB 00:02 2024-03-19T01:27:15.512 INFO:teuthology.orchestra.run.smithi144.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 997 kB/s | 1.8 MB 00:01 2024-03-19T01:27:15.654 INFO:teuthology.orchestra.run.smithi144.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 297 kB/s | 42 kB 00:00 2024-03-19T01:27:16.012 INFO:teuthology.orchestra.run.smithi144.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 693 kB/s | 248 kB 00:00 2024-03-19T01:27:16.129 INFO:teuthology.orchestra.run.smithi144.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 611 kB/s | 71 kB 00:00 2024-03-19T01:27:16.238 INFO:teuthology.orchestra.run.smithi144.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 337 kB/s | 36 kB 00:00 2024-03-19T01:27:16.472 INFO:teuthology.orchestra.run.smithi144.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 824 kB/s | 192 kB 00:00 2024-03-19T01:27:16.797 INFO:teuthology.orchestra.run.smithi144.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 787 kB/s | 256 kB 00:00 2024-03-19T01:27:16.914 INFO:teuthology.orchestra.run.smithi144.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 197 kB/s | 23 kB 00:00 2024-03-19T01:27:17.031 INFO:teuthology.orchestra.run.smithi144.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 929 kB/s | 108 kB 00:00 2024-03-19T01:27:17.231 INFO:teuthology.orchestra.run.smithi144.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 774 kB/s | 155 kB 00:00 2024-03-19T01:27:17.473 INFO:teuthology.orchestra.run.smithi144.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 561 kB/s | 136 kB 00:00 2024-03-19T01:27:17.598 INFO:teuthology.orchestra.run.smithi144.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 511 kB/s | 64 kB 00:00 2024-03-19T01:27:17.732 INFO:teuthology.orchestra.run.smithi144.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 215 kB/s | 29 kB 00:00 2024-03-19T01:27:19.232 INFO:teuthology.orchestra.run.smithi144.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 826 kB/s | 1.2 MB 00:01 2024-03-19T01:27:21.382 INFO:teuthology.orchestra.run.smithi144.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 1.0 MB/s | 2.2 MB 00:02 2024-03-19T01:27:23.240 INFO:teuthology.orchestra.run.smithi144.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 2.3 MB 00:01 2024-03-19T01:27:23.599 INFO:teuthology.orchestra.run.smithi144.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 1.3 MB/s | 465 kB 00:00 2024-03-19T01:27:23.707 INFO:teuthology.orchestra.run.smithi144.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 287 kB/s | 31 kB 00:00 2024-03-19T01:27:24.566 INFO:teuthology.orchestra.run.smithi144.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 1.3 MB/s | 1.1 MB 00:00 2024-03-19T01:27:24.907 INFO:teuthology.orchestra.run.smithi144.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 218 kB/s | 74 kB 00:00 2024-03-19T01:28:46.350 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 31 kB/s | 8.0 MB 04:28 2024-03-19T01:28:56.367 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 70 kB/s | 19 MB 04:34 2024-03-19T01:29:14.488 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T01:29:14.488 INFO:teuthology.orchestra.run.smithi144.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Connection timed out] 2024-03-19T01:29:14.847 INFO:teuthology.orchestra.run.smithi144.stdout:(173/174): lua-devel-5.4.4-4.el9.x86_64.rpm 177 B/s | 22 kB 02:09 2024-03-19T01:29:15.556 INFO:teuthology.orchestra.run.smithi144.stdout:(174/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.6 kB/s | 863 kB 02:09 2024-03-19T01:29:15.557 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-19T01:29:15.557 INFO:teuthology.orchestra.run.smithi144.stdout:Total 856 kB/s | 224 MB 04:28 2024-03-19T01:29:16.009 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-19T01:29:16.280 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-19T01:29:16.280 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-19T01:29:18.083 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-19T01:29:18.084 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-19T01:29:24.923 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-19T01:29:25.042 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-19T01:29:25.085 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-19T01:29:25.116 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T01:29:25.124 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T01:29:25.165 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-19T01:29:25.218 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-19T01:29:25.253 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-19T01:29:25.278 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-19T01:29:25.320 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-19T01:29:25.638 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-19T01:29:25.688 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T01:29:25.791 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T01:29:25.839 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T01:29:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T01:29:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T01:29:25.991 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-19T01:29:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-19T01:29:26.224 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-19T01:29:26.252 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T01:29:26.354 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T01:29:26.378 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T01:29:26.442 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T01:29:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-19T01:29:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T01:29:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T01:29:26.713 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T01:29:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T01:29:26.776 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-19T01:29:26.876 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-19T01:29:26.970 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T01:29:27.030 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T01:29:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T01:29:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T01:29:27.364 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T01:29:27.473 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-19T01:29:27.567 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-19T01:29:27.603 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-19T01:29:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-19T01:29:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-19T01:29:27.768 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-19T01:29:27.822 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-19T01:29:27.859 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-19T01:29:27.924 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-19T01:29:27.958 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-19T01:29:28.001 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-19T01:29:28.034 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-19T01:29:28.062 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-19T01:29:28.132 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-19T01:29:28.167 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-19T01:29:28.210 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-19T01:29:28.239 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-19T01:29:28.308 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-19T01:29:28.371 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-19T01:29:28.443 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-19T01:29:28.579 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-19T01:29:28.640 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-19T01:29:29.164 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-19T01:29:29.243 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-19T01:29:29.323 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-19T01:29:29.363 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T01:29:29.371 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T01:29:29.403 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-19T01:29:29.444 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-19T01:29:29.473 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-19T01:29:29.503 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-19T01:29:29.738 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-19T01:29:29.902 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-19T01:29:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-19T01:29:30.024 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-19T01:29:30.174 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-19T01:29:30.220 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-19T01:29:30.277 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-19T01:29:30.347 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-19T01:29:30.378 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-19T01:29:30.415 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-19T01:29:30.462 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-19T01:29:30.498 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-19T01:29:30.541 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-19T01:29:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-19T01:29:30.716 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-19T01:29:30.763 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-19T01:29:30.793 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-19T01:29:30.835 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-19T01:29:30.929 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-19T01:29:30.960 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-19T01:29:31.000 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-19T01:29:31.026 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-19T01:29:31.050 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-19T01:29:31.123 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-19T01:29:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-19T01:29:31.376 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-19T01:29:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-19T01:29:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-19T01:29:31.521 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-19T01:29:31.707 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-19T01:29:31.755 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-19T01:29:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-19T01:29:31.945 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-19T01:29:31.976 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-19T01:29:32.008 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-19T01:29:32.036 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-19T01:29:32.078 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-19T01:29:32.117 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-19T01:29:32.149 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-19T01:29:32.179 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-19T01:29:32.256 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-19T01:29:32.289 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-19T01:29:32.325 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-19T01:29:32.364 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-19T01:29:32.399 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-19T01:29:32.488 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-19T01:29:32.579 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-19T01:29:32.621 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-19T01:29:32.653 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-19T01:29:32.700 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-19T01:29:32.798 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-19T01:29:32.855 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-19T01:29:33.584 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-19T01:29:33.635 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-19T01:29:33.677 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-19T01:29:33.725 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-19T01:29:33.814 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-19T01:29:33.861 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-19T01:29:34.038 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-19T01:29:34.076 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-19T01:29:34.153 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-19T01:29:34.784 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-19T01:29:34.906 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-19T01:29:36.119 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-19T01:29:36.202 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-19T01:29:36.232 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-19T01:29:36.255 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-19T01:29:36.274 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-19T01:29:36.585 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-19T01:29:36.613 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T01:29:36.703 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T01:29:36.734 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-19T01:29:36.770 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-19T01:29:37.231 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-19T01:29:37.257 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T01:29:37.328 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T01:29:37.351 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-19T01:29:39.597 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T01:29:39.647 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T01:29:40.184 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T01:29:40.218 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T01:29:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T01:29:40.299 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-19T01:29:40.299 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-19T01:29:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:40.327 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T01:29:52.332 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T01:29:52.333 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-03-19T01:29:52.333 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-03-19T01:29:52.333 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-03-19T01:29:52.333 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-03-19T01:29:52.333 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-03-19T01:29:52.333 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-03-19T01:29:52.333 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-03-19T01:29:52.333 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:52.491 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T01:29:52.506 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T01:29:52.506 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:29:52.506 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-19T01:29:52.506 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T01:29:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T01:29:54.019 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:29:54.020 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-19T01:29:54.020 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T01:29:54.020 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T01:29:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T01:29:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T01:29:54.290 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:29:54.290 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-19T01:29:54.290 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T01:29:54.290 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T01:29:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:54.747 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T01:29:54.769 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T01:29:54.770 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:29:54.770 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-19T01:29:54.770 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T01:29:54.770 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T01:29:54.770 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:54.866 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-19T01:29:55.266 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T01:29:55.297 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T01:29:55.321 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-19T01:29:55.372 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-19T01:29:55.395 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T01:29:55.626 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T01:29:55.655 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T01:29:56.612 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T01:29:56.638 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T01:29:56.738 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T01:29:56.908 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-19T01:29:56.932 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T01:29:56.955 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T01:29:56.955 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:29:56.955 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-19T01:29:56.955 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T01:29:56.955 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T01:29:56.955 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:56.999 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T01:29:57.011 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T01:29:57.056 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-19T01:29:57.930 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-19T01:29:57.969 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T01:29:57.990 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T01:29:57.991 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:29:57.991 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-19T01:29:57.991 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T01:29:57.991 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T01:29:57.991 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:58.035 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T01:29:58.057 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T01:29:58.057 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:29:58.057 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-19T01:29:58.057 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:58.103 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T01:29:58.125 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T01:29:58.125 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:29:58.125 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-19T01:29:58.125 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T01:29:58.125 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T01:29:58.126 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:29:58.436 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T01:29:58.459 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T01:29:58.459 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:29:58.459 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-19T01:29:58.459 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T01:29:58.459 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T01:29:58.459 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:30:02.891 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T01:30:02.932 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-19T01:30:02.962 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-19T01:30:02.981 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-19T01:30:03.162 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-19T01:30:03.274 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-19T01:30:03.313 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-19T01:30:03.353 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-19T01:30:03.405 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-19T01:30:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-19T01:30:03.481 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-19T01:30:03.513 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-19T01:30:03.543 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-19T01:30:03.606 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-19T01:30:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-19T01:30:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-19T01:30:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T01:30:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T01:30:04.377 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T01:30:05.210 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-19T01:30:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-19T01:30:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-19T01:30:05.433 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-19T01:30:05.481 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-19T01:30:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-19T01:30:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-19T01:30:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-19T01:30:05.558 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T01:30:05.653 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T01:30:05.683 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-19T01:30:05.683 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T01:30:05.768 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T01:30:05.794 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-19T01:30:05.836 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-19T01:30:05.861 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-19T01:30:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-19T01:30:05.919 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-19T01:30:05.958 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-19T01:30:05.991 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-19T01:30:06.016 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-19T01:30:06.041 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-19T01:30:06.066 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-19T01:30:06.092 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-19T01:30:06.098 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T01:30:06.098 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T01:30:06.278 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T01:30:06.278 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T01:30:06.361 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T01:30:06.386 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-19T01:30:06.411 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-19T01:30:06.436 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-19T01:30:06.462 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-19T01:30:06.526 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T01:30:11.330 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T01:30:11.330 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-19T01:30:11.330 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-19T01:30:11.330 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-19T01:30:11.330 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-19T01:30:11.330 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T01:30:11.331 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-19T01:30:11.332 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-19T01:30:11.333 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-19T01:30:11.334 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-19T01:30:11.334 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-19T01:30:11.334 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-19T01:30:11.334 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-19T01:30:11.334 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-19T01:30:11.334 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-19T01:30:11.334 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-19T01:30:11.334 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-19T01:30:11.334 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-19T01:30:11.335 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-19T01:30:11.336 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-19T01:30:11.337 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-19T01:30:11.338 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-19T01:30:11.339 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-19T01:30:11.340 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-19T01:30:11.341 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-19T01:30:11.342 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-19T01:30:11.343 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-19T01:30:11.344 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-19T01:30:11.345 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-19T01:30:11.345 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-19T01:30:11.345 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-19T01:30:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-19T01:30:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: acl-2.3.1-4.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-19T01:30:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-19T01:30:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: jq-1.6-16.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-19T01:30:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.11-1.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-19T01:30:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-19T01:30:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-19T01:30:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-19T01:30:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-19T01:30:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-19T01:30:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-19T01:30:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-19T01:30:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-19T01:30:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-19T01:30:14.512 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-19T01:30:14.983 DEBUG:teuthology.parallel:result is None 2024-03-19T01:33:26.061 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 69 kB/s | 19 MB 04:38 2024-03-19T01:33:26.603 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 24 kB/s | 6.1 MB 04:26 2024-03-19T01:33:29.986 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-19T01:37:51.406 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 60 B/s | 16 kB 04:23 2024-03-19T01:37:53.140 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-19T01:37:53.650 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 61 B/s | 16 kB 04:22 2024-03-19T01:37:55.012 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-19T01:37:58.056 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-19T01:37:59.885 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-19T01:38:00.370 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T01:38:00.371 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T01:38:00.384 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-19T01:38:00.385 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-19T01:38:00.386 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-19T01:38:00.387 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-19T01:38:00.389 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-19T01:38:00.392 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-19T01:38:00.393 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-19T01:38:00.472 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-03-19T01:38:00.492 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================================ 2024-03-19T01:38:00.492 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-03-19T01:38:00.492 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================================ 2024-03-19T01:38:00.492 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-03-19T01:38:00.492 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-19T01:38:00.493 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-19T01:38:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-19T01:38:00.495 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-19T01:38:00.496 INFO:teuthology.orchestra.run.smithi092.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-19T01:38:00.497 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-19T01:38:00.498 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-19T01:38:00.499 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-19T01:38:00.500 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-19T01:38:00.501 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-19T01:38:00.502 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-19T01:38:00.503 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-19T01:38:00.504 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-19T01:38:00.505 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================================ 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout:Install 150 Packages 2024-03-19T01:38:00.506 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 24 Packages 2024-03-19T01:38:00.507 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:38:00.507 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 224 M 2024-03-19T01:38:00.507 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-03-19T01:38:01.324 INFO:teuthology.orchestra.run.smithi092.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-03-19T01:38:01.516 INFO:teuthology.orchestra.run.smithi092.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 4.5 MB/s | 891 kB 00:00 2024-03-19T01:38:01.591 INFO:teuthology.orchestra.run.smithi092.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 1.9 MB/s | 145 kB 00:00 2024-03-19T01:38:01.942 INFO:teuthology.orchestra.run.smithi092.stdout:(4/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 6.7 MB/s | 2.4 MB 00:00 2024-03-19T01:38:02.184 INFO:teuthology.orchestra.run.smithi092.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 6.2 MB/s | 1.5 MB 00:00 2024-03-19T01:38:02.229 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T01:38:02.230 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-19T01:38:02.242 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-19T01:38:02.243 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-19T01:38:02.244 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-19T01:38:02.245 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-19T01:38:02.246 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-19T01:38:02.249 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-19T01:38:02.250 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-19T01:38:02.329 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-03-19T01:38:02.344 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 6.6 k 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.4 M 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 891 k 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 145 k 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 1.5 M 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 139 k 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 1.8 M 2024-03-19T01:38:02.345 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 7.4 M 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 49 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 9.8 M 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 48 M 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 264 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 749 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 221 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 27 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 18 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 32 k 2024-03-19T01:38:02.346 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 746 k 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 127 k 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 162 k 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 321 k 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 298 k 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 99 k 2024-03-19T01:38:02.347 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 85 k 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 170 k 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-19T01:38:02.348 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.4 M 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 3.1 M 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-19T01:38:02.349 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 21 M 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 34 k 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 2.4 M 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 248 k 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 4.7 M 2024-03-19T01:38:02.350 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 17 M 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2206.gb6ae2491.el9 ceph-noarch 16 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 25 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 163 k 2024-03-19T01:38:02.351 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 501 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 5.1 M 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-19T01:38:02.352 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-19T01:38:02.353 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 45 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.0.0-2206.gb6ae2491.el9 ceph 131 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-19T01:38:02.354 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-19T01:38:02.355 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-19T01:38:02.356 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-19T01:38:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout:Installing weak dependencies: 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-19T01:38:02.358 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:38:02.359 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-03-19T01:38:02.359 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-03-19T01:38:02.359 INFO:teuthology.orchestra.run.smithi159.stdout:Install 150 Packages 2024-03-19T01:38:02.359 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 24 Packages 2024-03-19T01:38:02.359 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:38:02.359 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 224 M 2024-03-19T01:38:02.360 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-03-19T01:38:02.709 INFO:teuthology.orchestra.run.smithi092.stdout:(6/174): ceph-common-19.0.0-2206.gb6ae2491.el9. 13 MB/s | 21 MB 00:01 2024-03-19T01:38:02.792 INFO:teuthology.orchestra.run.smithi092.stdout:(7/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 7.7 MB/s | 4.7 MB 00:00 2024-03-19T01:38:03.661 INFO:teuthology.orchestra.run.smithi159.stdout:(1/174): ceph-19.0.0-2206.gb6ae2491.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-03-19T01:38:03.793 INFO:teuthology.orchestra.run.smithi092.stdout:(8/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el9 9.8 MB/s | 9.8 MB 00:00 2024-03-19T01:38:03.959 INFO:teuthology.orchestra.run.smithi092.stdout:(9/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x86 13 MB/s | 17 MB 00:01 2024-03-19T01:38:03.984 INFO:teuthology.orchestra.run.smithi092.stdout:(10/174): ceph-selinux-19.0.0-2206.gb6ae2491.el 131 kB/s | 25 kB 00:00 2024-03-19T01:38:04.193 INFO:teuthology.orchestra.run.smithi092.stdout:(11/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x 1.8 MB/s | 5.4 MB 00:03 2024-03-19T01:38:04.293 INFO:teuthology.orchestra.run.smithi092.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 271 kB/s | 27 kB 00:00 2024-03-19T01:38:04.386 INFO:teuthology.orchestra.run.smithi159.stdout:(2/174): ceph-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.2 MB/s | 891 kB 00:00 2024-03-19T01:38:04.402 INFO:teuthology.orchestra.run.smithi092.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 293 kB/s | 32 kB 00:00 2024-03-19T01:38:04.577 INFO:teuthology.orchestra.run.smithi092.stdout:(14/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 373 kB/s | 221 kB 00:00 2024-03-19T01:38:04.586 INFO:teuthology.orchestra.run.smithi159.stdout:(3/174): ceph-immutable-object-cache-19.0.0-220 724 kB/s | 145 kB 00:00 2024-03-19T01:38:04.669 INFO:teuthology.orchestra.run.smithi092.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 1.7 MB/s | 163 kB 00:00 2024-03-19T01:38:04.761 INFO:teuthology.orchestra.run.smithi092.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 1.4 MB/s | 127 kB 00:00 2024-03-19T01:38:04.870 INFO:teuthology.orchestra.run.smithi092.stdout:(17/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 1.6 MB/s | 746 kB 00:00 2024-03-19T01:38:05.337 INFO:teuthology.orchestra.run.smithi159.stdout:(4/174): ceph-common-19.0.0-2206.gb6ae2491.el9. 9.9 MB/s | 21 MB 00:02 2024-03-19T01:38:05.362 INFO:teuthology.orchestra.run.smithi092.stdout:(18/174): libradosstriper1-19.0.0-2206.gb6ae249 834 kB/s | 501 kB 00:00 2024-03-19T01:38:05.437 INFO:teuthology.orchestra.run.smithi092.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 604 kB/s | 45 kB 00:00 2024-03-19T01:38:05.495 INFO:teuthology.orchestra.run.smithi159.stdout:(5/174): ceph-mgr-19.0.0-2206.gb6ae2491.el9.x86 9.5 MB/s | 1.5 MB 00:00 2024-03-19T01:38:05.496 INFO:teuthology.orchestra.run.smithi092.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 2.2 MB/s | 131 kB 00:00 2024-03-19T01:38:05.596 INFO:teuthology.orchestra.run.smithi092.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 1.6 MB/s | 162 kB 00:00 2024-03-19T01:38:05.688 INFO:teuthology.orchestra.run.smithi092.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 3.4 MB/s | 321 kB 00:00 2024-03-19T01:38:05.780 INFO:teuthology.orchestra.run.smithi092.stdout:(23/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 3.2 MB/s | 298 kB 00:00 2024-03-19T01:38:05.864 INFO:teuthology.orchestra.run.smithi092.stdout:(24/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.2 MB/s | 99 kB 00:00 2024-03-19T01:38:05.922 INFO:teuthology.orchestra.run.smithi092.stdout:(25/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-19T01:38:05.929 INFO:teuthology.orchestra.run.smithi159.stdout:(6/174): ceph-mon-19.0.0-2206.gb6ae2491.el9.x86 11 MB/s | 4.7 MB 00:00 2024-03-19T01:38:06.004 INFO:teuthology.orchestra.run.smithi159.stdout:(7/174): ceph-base-19.0.0-2206.gb6ae2491.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-03-19T01:38:06.188 INFO:teuthology.orchestra.run.smithi159.stdout:(8/174): ceph-mds-19.0.0-2206.gb6ae2491.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-03-19T01:38:06.297 INFO:teuthology.orchestra.run.smithi159.stdout:(9/174): ceph-selinux-19.0.0-2206.gb6ae2491.el9 232 kB/s | 25 kB 00:00 2024-03-19T01:38:06.481 INFO:teuthology.orchestra.run.smithi092.stdout:(26/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 5.5 MB/s | 3.1 MB 00:00 2024-03-19T01:38:06.573 INFO:teuthology.orchestra.run.smithi092.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-19T01:38:06.664 INFO:teuthology.orchestra.run.smithi092.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 371 kB/s | 34 kB 00:00 2024-03-19T01:38:06.756 INFO:teuthology.orchestra.run.smithi092.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 1.5 MB/s | 139 kB 00:00 2024-03-19T01:38:06.840 INFO:teuthology.orchestra.run.smithi092.stdout:(30/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 2.6 MB/s | 5.1 MB 00:01 2024-03-19T01:38:07.149 INFO:teuthology.orchestra.run.smithi092.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 4.6 MB/s | 1.8 MB 00:00 2024-03-19T01:38:07.241 INFO:teuthology.orchestra.run.smithi092.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2206.gb6 2.7 MB/s | 248 kB 00:00 2024-03-19T01:38:07.324 INFO:teuthology.orchestra.run.smithi092.stdout:(33/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 592 kB/s | 49 kB 00:00 2024-03-19T01:38:07.400 INFO:teuthology.orchestra.run.smithi092.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2206.gb 219 kB/s | 16 kB 00:00 2024-03-19T01:38:07.466 INFO:teuthology.orchestra.run.smithi092.stdout:(35/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 3.9 MB/s | 264 kB 00:00 2024-03-19T01:38:07.539 INFO:teuthology.orchestra.run.smithi159.stdout:(10/174): ceph-osd-19.0.0-2206.gb6ae2491.el9.x8 10 MB/s | 17 MB 00:01 2024-03-19T01:38:07.575 INFO:teuthology.orchestra.run.smithi092.stdout:(36/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 6.8 MB/s | 749 kB 00:00 2024-03-19T01:38:07.614 INFO:teuthology.orchestra.run.smithi159.stdout:(11/174): cephfs-mirror-19.0.0-2206.gb6ae2491.e 2.9 MB/s | 221 kB 00:00 2024-03-19T01:38:07.625 INFO:teuthology.orchestra.run.smithi092.stdout:(37/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 366 kB/s | 18 kB 00:00 2024-03-19T01:38:07.672 INFO:teuthology.orchestra.run.smithi159.stdout:(12/174): cephfs-shell-19.0.0-2206.gb6ae2491.el 465 kB/s | 27 kB 00:00 2024-03-19T01:38:07.731 INFO:teuthology.orchestra.run.smithi159.stdout:(13/174): libcephfs-devel-19.0.0-2206.gb6ae2491 544 kB/s | 32 kB 00:00 2024-03-19T01:38:07.759 INFO:teuthology.orchestra.run.smithi092.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 662 kB 00:00 2024-03-19T01:38:07.809 INFO:teuthology.orchestra.run.smithi092.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-03-19T01:38:07.814 INFO:teuthology.orchestra.run.smithi159.stdout:(14/174): libcephfs2-19.0.0-2206.gb6ae2491.el9. 8.8 MB/s | 746 kB 00:00 2024-03-19T01:38:07.843 INFO:teuthology.orchestra.run.smithi092.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-19T01:38:07.873 INFO:teuthology.orchestra.run.smithi159.stdout:(15/174): libcephsqlite-19.0.0-2206.gb6ae2491.e 2.7 MB/s | 163 kB 00:00 2024-03-19T01:38:07.885 INFO:teuthology.orchestra.run.smithi092.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-19T01:38:07.918 INFO:teuthology.orchestra.run.smithi092.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 948 kB/s | 32 kB 00:00 2024-03-19T01:38:07.923 INFO:teuthology.orchestra.run.smithi159.stdout:(16/174): librados-devel-19.0.0-2206.gb6ae2491. 2.5 MB/s | 127 kB 00:00 2024-03-19T01:38:07.999 INFO:teuthology.orchestra.run.smithi159.stdout:(17/174): libradosstriper1-19.0.0-2206.gb6ae249 6.5 MB/s | 501 kB 00:00 2024-03-19T01:38:08.019 INFO:teuthology.orchestra.run.smithi092.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-19T01:38:08.053 INFO:teuthology.orchestra.run.smithi092.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 516 kB/s | 17 kB 00:00 2024-03-19T01:38:08.086 INFO:teuthology.orchestra.run.smithi092.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-03-19T01:38:08.120 INFO:teuthology.orchestra.run.smithi092.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-03-19T01:38:08.154 INFO:teuthology.orchestra.run.smithi092.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-19T01:38:08.204 INFO:teuthology.orchestra.run.smithi092.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-19T01:38:08.246 INFO:teuthology.orchestra.run.smithi092.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-03-19T01:38:08.296 INFO:teuthology.orchestra.run.smithi092.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 933 kB/s | 46 kB 00:00 2024-03-19T01:38:08.338 INFO:teuthology.orchestra.run.smithi092.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-19T01:38:08.380 INFO:teuthology.orchestra.run.smithi092.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-03-19T01:38:08.422 INFO:teuthology.orchestra.run.smithi092.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-03-19T01:38:08.441 INFO:teuthology.orchestra.run.smithi159.stdout:(18/174): librgw2-19.0.0-2206.gb6ae2491.el9.x86 12 MB/s | 5.1 MB 00:00 2024-03-19T01:38:08.456 INFO:teuthology.orchestra.run.smithi092.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-19T01:38:08.483 INFO:teuthology.orchestra.run.smithi159.stdout:(19/174): python3-ceph-argparse-19.0.0-2206.gb6 1.1 MB/s | 45 kB 00:00 2024-03-19T01:38:08.533 INFO:teuthology.orchestra.run.smithi159.stdout:(20/174): python3-ceph-common-19.0.0-2206.gb6ae 2.6 MB/s | 131 kB 00:00 2024-03-19T01:38:08.583 INFO:teuthology.orchestra.run.smithi159.stdout:(21/174): python3-cephfs-19.0.0-2206.gb6ae2491. 3.2 MB/s | 162 kB 00:00 2024-03-19T01:38:08.597 INFO:teuthology.orchestra.run.smithi092.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-03-19T01:38:08.642 INFO:teuthology.orchestra.run.smithi159.stdout:(22/174): python3-rados-19.0.0-2206.gb6ae2491.e 5.4 MB/s | 321 kB 00:00 2024-03-19T01:38:08.656 INFO:teuthology.orchestra.run.smithi092.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 26 MB/s | 1.5 MB 00:00 2024-03-19T01:38:08.706 INFO:teuthology.orchestra.run.smithi092.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 16 MB/s | 811 kB 00:00 2024-03-19T01:38:08.717 INFO:teuthology.orchestra.run.smithi159.stdout:(23/174): python3-rbd-19.0.0-2206.gb6ae2491.el9 3.9 MB/s | 298 kB 00:00 2024-03-19T01:38:08.750 INFO:teuthology.orchestra.run.smithi092.stdout:(58/174): postgresql-private-libs-13.11-1.el9.x 3.1 MB/s | 134 kB 00:00 2024-03-19T01:38:08.776 INFO:teuthology.orchestra.run.smithi159.stdout:(24/174): python3-rgw-19.0.0-2206.gb6ae2491.el9 1.7 MB/s | 99 kB 00:00 2024-03-19T01:38:08.835 INFO:teuthology.orchestra.run.smithi159.stdout:(25/174): rbd-fuse-19.0.0-2206.gb6ae2491.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-19T01:38:08.907 INFO:teuthology.orchestra.run.smithi092.stdout:(59/174): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-19T01:38:09.060 INFO:teuthology.orchestra.run.smithi159.stdout:(26/174): rbd-mirror-19.0.0-2206.gb6ae2491.el9. 14 MB/s | 3.1 MB 00:00 2024-03-19T01:38:09.110 INFO:teuthology.orchestra.run.smithi159.stdout:(27/174): rbd-nbd-19.0.0-2206.gb6ae2491.el9.x86 3.3 MB/s | 170 kB 00:00 2024-03-19T01:38:09.152 INFO:teuthology.orchestra.run.smithi159.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2206.g 814 kB/s | 34 kB 00:00 2024-03-19T01:38:09.202 INFO:teuthology.orchestra.run.smithi159.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2206.gb6ae249 2.7 MB/s | 139 kB 00:00 2024-03-19T01:38:09.324 INFO:teuthology.orchestra.run.smithi092.stdout:(60/174): ceph-test-19.0.0-2206.gb6ae2491.el9.x 8.9 MB/s | 48 MB 00:05 2024-03-19T01:38:09.344 INFO:teuthology.orchestra.run.smithi159.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2206.gb6ae2 13 MB/s | 1.8 MB 00:00 2024-03-19T01:38:09.358 INFO:teuthology.orchestra.run.smithi092.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 2.2 MB/s | 1.0 MB 00:00 2024-03-19T01:38:09.383 INFO:teuthology.orchestra.run.smithi092.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-19T01:38:09.517 INFO:teuthology.orchestra.run.smithi092.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-03-19T01:38:09.533 INFO:teuthology.orchestra.run.smithi092.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-03-19T01:38:09.559 INFO:teuthology.orchestra.run.smithi092.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 434 kB/s | 18 kB 00:00 2024-03-19T01:38:09.643 INFO:teuthology.orchestra.run.smithi092.stdout:(66/174): ceph-mgr-diskprediction-local-19.0.0- 2.6 MB/s | 7.4 MB 00:02 2024-03-19T01:38:09.668 INFO:teuthology.orchestra.run.smithi092.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2024-03-19T01:38:09.693 INFO:teuthology.orchestra.run.smithi092.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 236 kB/s | 32 kB 00:00 2024-03-19T01:38:09.718 INFO:teuthology.orchestra.run.smithi092.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 217 kB 00:00 2024-03-19T01:38:09.803 INFO:teuthology.orchestra.run.smithi159.stdout:(31/174): ceph-radosgw-19.0.0-2206.gb6ae2491.el 2.6 MB/s | 9.8 MB 00:03 2024-03-19T01:38:09.838 INFO:teuthology.orchestra.run.smithi092.stdout:(70/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 4.8 MB 00:00 2024-03-19T01:38:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 996 kB/s | 174 kB 00:00 2024-03-19T01:38:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 397 kB/s | 69 kB 00:00 2024-03-19T01:38:09.903 INFO:teuthology.orchestra.run.smithi159.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-03-19T01:38:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 511 kB/s | 41 kB 00:00 2024-03-19T01:38:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-19T01:38:09.945 INFO:teuthology.orchestra.run.smithi159.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2206.gb6 1.7 MB/s | 248 kB 00:00 2024-03-19T01:38:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:(75/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-19T01:38:09.971 INFO:teuthology.orchestra.run.smithi159.stdout:(34/174): ceph-mgr-rook-19.0.0-2206.gb6ae2491.e 739 kB/s | 49 kB 00:00 2024-03-19T01:38:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-19T01:38:10.004 INFO:teuthology.orchestra.run.smithi159.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2206.gb 281 kB/s | 16 kB 00:00 2024-03-19T01:38:10.020 INFO:teuthology.orchestra.run.smithi092.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 636 kB/s | 47 kB 00:00 2024-03-19T01:38:10.031 INFO:teuthology.orchestra.run.smithi159.stdout:(36/174): ceph-volume-19.0.0-2206.gb6ae2491.el9 4.2 MB/s | 264 kB 00:00 2024-03-19T01:38:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 858 kB/s | 43 kB 00:00 2024-03-19T01:38:10.065 INFO:teuthology.orchestra.run.smithi159.stdout:(37/174): cephfs-top-19.0.0-2206.gb6ae2491.el9. 548 kB/s | 18 kB 00:00 2024-03-19T01:38:10.204 INFO:teuthology.orchestra.run.smithi092.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 212 kB/s | 39 kB 00:00 2024-03-19T01:38:10.299 INFO:teuthology.orchestra.run.smithi159.stdout:(38/174): cephadm-19.0.0-2206.gb6ae2491.el9.noa 2.5 MB/s | 749 kB 00:00 2024-03-19T01:38:10.416 INFO:teuthology.orchestra.run.smithi159.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2024-03-19T01:38:10.438 INFO:teuthology.orchestra.run.smithi092.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-19T01:38:10.458 INFO:teuthology.orchestra.run.smithi159.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.7 MB/s | 920 kB 00:00 2024-03-19T01:38:10.463 INFO:teuthology.orchestra.run.smithi092.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 99 kB/s | 41 kB 00:00 2024-03-19T01:38:10.483 INFO:teuthology.orchestra.run.smithi159.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-19T01:38:10.488 INFO:teuthology.orchestra.run.smithi092.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-03-19T01:38:10.514 INFO:teuthology.orchestra.run.smithi092.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 756 kB/s | 56 kB 00:00 2024-03-19T01:38:10.516 INFO:teuthology.orchestra.run.smithi159.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-19T01:38:10.533 INFO:teuthology.orchestra.run.smithi159.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-03-19T01:38:10.539 INFO:teuthology.orchestra.run.smithi092.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-03-19T01:38:10.564 INFO:teuthology.orchestra.run.smithi092.stdout:(85/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 831 kB/s | 63 kB 00:00 2024-03-19T01:38:10.589 INFO:teuthology.orchestra.run.smithi092.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-19T01:38:10.592 INFO:teuthology.orchestra.run.smithi159.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 294 kB/s | 17 kB 00:00 2024-03-19T01:38:10.615 INFO:teuthology.orchestra.run.smithi092.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-19T01:38:10.640 INFO:teuthology.orchestra.run.smithi092.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-03-19T01:38:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-03-19T01:38:10.673 INFO:teuthology.orchestra.run.smithi092.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-19T01:38:10.676 INFO:teuthology.orchestra.run.smithi159.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 123 kB/s | 10 kB 00:00 2024-03-19T01:38:10.690 INFO:teuthology.orchestra.run.smithi092.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 124 kB 00:00 2024-03-19T01:38:10.701 INFO:teuthology.orchestra.run.smithi159.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-19T01:38:10.716 INFO:teuthology.orchestra.run.smithi092.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 187 kB 00:00 2024-03-19T01:38:10.718 INFO:teuthology.orchestra.run.smithi159.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-19T01:38:10.761 INFO:teuthology.orchestra.run.smithi159.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-19T01:38:10.794 INFO:teuthology.orchestra.run.smithi159.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-19T01:38:10.811 INFO:teuthology.orchestra.run.smithi159.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 46 kB 00:00 2024-03-19T01:38:10.861 INFO:teuthology.orchestra.run.smithi159.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2024-03-19T01:38:10.887 INFO:teuthology.orchestra.run.smithi159.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-19T01:38:10.937 INFO:teuthology.orchestra.run.smithi159.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-19T01:38:10.962 INFO:teuthology.orchestra.run.smithi159.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 453 kB/s | 34 kB 00:00 2024-03-19T01:38:10.974 INFO:teuthology.orchestra.run.smithi092.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 757 kB/s | 195 kB 00:00 2024-03-19T01:38:10.991 INFO:teuthology.orchestra.run.smithi092.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 591 kB/s | 187 kB 00:00 2024-03-19T01:38:11.058 INFO:teuthology.orchestra.run.smithi092.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-03-19T01:38:11.119 INFO:teuthology.orchestra.run.smithi092.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 169 kB/s | 21 kB 00:00 2024-03-19T01:38:11.163 INFO:teuthology.orchestra.run.smithi159.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-03-19T01:38:11.177 INFO:teuthology.orchestra.run.smithi092.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 216 kB/s | 25 kB 00:00 2024-03-19T01:38:11.211 INFO:teuthology.orchestra.run.smithi092.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-03-19T01:38:11.221 INFO:teuthology.orchestra.run.smithi159.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 5.9 MB/s | 1.5 MB 00:00 2024-03-19T01:38:11.255 INFO:teuthology.orchestra.run.smithi159.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 8.8 MB/s | 811 kB 00:00 2024-03-19T01:38:11.280 INFO:teuthology.orchestra.run.smithi159.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-19T01:38:11.336 INFO:teuthology.orchestra.run.smithi092.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-03-19T01:38:11.362 INFO:teuthology.orchestra.run.smithi092.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-03-19T01:38:11.472 INFO:teuthology.orchestra.run.smithi159.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2024-03-19T01:38:11.495 INFO:teuthology.orchestra.run.smithi092.stdout:(100/174): openssl-devel-3.0.7-27.el9.x86_64.rp 11 MB/s | 4.1 MB 00:00 2024-03-19T01:38:11.505 INFO:teuthology.orchestra.run.smithi159.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-03-19T01:38:11.521 INFO:teuthology.orchestra.run.smithi092.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 259 kB/s | 35 kB 00:00 2024-03-19T01:38:11.531 INFO:teuthology.orchestra.run.smithi159.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-19T01:38:11.571 INFO:teuthology.orchestra.run.smithi092.stdout:(102/174): qatlib-23.11.0-1.el9.x86_64.rpm 924 kB/s | 216 kB 00:00 2024-03-19T01:38:11.621 INFO:teuthology.orchestra.run.smithi092.stdout:(103/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 355 kB/s | 44 kB 00:00 2024-03-19T01:38:11.680 INFO:teuthology.orchestra.run.smithi092.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 694 kB/s | 94 kB 00:00 2024-03-19T01:38:11.689 INFO:teuthology.orchestra.run.smithi159.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 5.8 MB 00:00 2024-03-19T01:38:11.705 INFO:teuthology.orchestra.run.smithi092.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 203 kB/s | 22 kB 00:00 2024-03-19T01:38:11.723 INFO:teuthology.orchestra.run.smithi159.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 128 kB/s | 24 kB 00:00 2024-03-19T01:38:11.740 INFO:teuthology.orchestra.run.smithi159.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 360 kB/s | 18 kB 00:00 2024-03-19T01:38:11.766 INFO:teuthology.orchestra.run.smithi159.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-19T01:38:11.782 INFO:teuthology.orchestra.run.smithi159.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 756 kB/s | 32 kB 00:00 2024-03-19T01:38:11.905 INFO:teuthology.orchestra.run.smithi092.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-03-19T01:38:11.941 INFO:teuthology.orchestra.run.smithi159.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-03-19T01:38:11.966 INFO:teuthology.orchestra.run.smithi159.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-03-19T01:38:11.992 INFO:teuthology.orchestra.run.smithi159.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-19T01:38:12.017 INFO:teuthology.orchestra.run.smithi159.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-19T01:38:12.034 INFO:teuthology.orchestra.run.smithi159.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 981 kB/s | 41 kB 00:00 2024-03-19T01:38:12.059 INFO:teuthology.orchestra.run.smithi159.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-03-19T01:38:12.085 INFO:teuthology.orchestra.run.smithi159.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-19T01:38:12.110 INFO:teuthology.orchestra.run.smithi159.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-19T01:38:12.127 INFO:teuthology.orchestra.run.smithi159.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-19T01:38:12.144 INFO:teuthology.orchestra.run.smithi159.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-19T01:38:12.244 INFO:teuthology.orchestra.run.smithi159.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 390 kB/s | 39 kB 00:00 2024-03-19T01:38:12.461 INFO:teuthology.orchestra.run.smithi159.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-19T01:38:12.478 INFO:teuthology.orchestra.run.smithi159.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 177 kB/s | 41 kB 00:00 2024-03-19T01:38:12.528 INFO:teuthology.orchestra.run.smithi159.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-03-19T01:38:12.545 INFO:teuthology.orchestra.run.smithi159.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 844 kB/s | 56 kB 00:00 2024-03-19T01:38:12.579 INFO:teuthology.orchestra.run.smithi159.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-19T01:38:12.597 INFO:teuthology.orchestra.run.smithi159.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 874 kB/s | 44 kB 00:00 2024-03-19T01:38:12.638 INFO:teuthology.orchestra.run.smithi159.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-19T01:38:12.680 INFO:teuthology.orchestra.run.smithi159.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-03-19T01:38:12.730 INFO:teuthology.orchestra.run.smithi159.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-19T01:38:12.755 INFO:teuthology.orchestra.run.smithi159.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-19T01:38:12.797 INFO:teuthology.orchestra.run.smithi159.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 124 kB 00:00 2024-03-19T01:38:12.823 INFO:teuthology.orchestra.run.smithi159.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-03-19T01:38:12.989 INFO:teuthology.orchestra.run.smithi092.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 419 kB/s | 548 kB 00:01 2024-03-19T01:38:13.090 INFO:teuthology.orchestra.run.smithi092.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 16 kB/s | 19 kB 00:01 2024-03-19T01:38:13.148 INFO:teuthology.orchestra.run.smithi092.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-03-19T01:38:13.206 INFO:teuthology.orchestra.run.smithi092.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-03-19T01:38:13.265 INFO:teuthology.orchestra.run.smithi092.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-19T01:38:13.373 INFO:teuthology.orchestra.run.smithi159.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 326 kB/s | 187 kB 00:00 2024-03-19T01:38:13.399 INFO:teuthology.orchestra.run.smithi092.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-19T01:38:13.424 INFO:teuthology.orchestra.run.smithi092.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 953 kB/s | 151 kB 00:00 2024-03-19T01:38:13.491 INFO:teuthology.orchestra.run.smithi092.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-03-19T01:38:13.516 INFO:teuthology.orchestra.run.smithi092.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-03-19T01:38:13.550 INFO:teuthology.orchestra.run.smithi092.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-03-19T01:38:13.575 INFO:teuthology.orchestra.run.smithi092.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 548 kB/s | 32 kB 00:00 2024-03-19T01:38:13.607 INFO:teuthology.orchestra.run.smithi159.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 836 kB/s | 195 kB 00:00 2024-03-19T01:38:13.608 INFO:teuthology.orchestra.run.smithi092.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-03-19T01:38:13.634 INFO:teuthology.orchestra.run.smithi092.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-19T01:38:13.673 INFO:teuthology.orchestra.run.smithi159.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 944 kB/s | 803 kB 00:00 2024-03-19T01:38:13.701 INFO:teuthology.orchestra.run.smithi092.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-19T01:38:13.726 INFO:teuthology.orchestra.run.smithi092.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-03-19T01:38:13.732 INFO:teuthology.orchestra.run.smithi159.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 996 kB/s | 125 kB 00:00 2024-03-19T01:38:13.768 INFO:teuthology.orchestra.run.smithi092.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 528 kB/s | 35 kB 00:00 2024-03-19T01:38:13.783 INFO:teuthology.orchestra.run.smithi159.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-03-19T01:38:13.793 INFO:teuthology.orchestra.run.smithi092.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-19T01:38:13.833 INFO:teuthology.orchestra.run.smithi159.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-03-19T01:38:13.852 INFO:teuthology.orchestra.run.smithi092.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-19T01:38:13.902 INFO:teuthology.orchestra.run.smithi092.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-03-19T01:38:13.952 INFO:teuthology.orchestra.run.smithi092.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-03-19T01:38:13.958 INFO:teuthology.orchestra.run.smithi159.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-03-19T01:38:13.969 INFO:teuthology.orchestra.run.smithi092.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-03-19T01:38:14.003 INFO:teuthology.orchestra.run.smithi092.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-03-19T01:38:14.020 INFO:teuthology.orchestra.run.smithi092.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-03-19T01:38:14.062 INFO:teuthology.orchestra.run.smithi092.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-03-19T01:38:14.100 INFO:teuthology.orchestra.run.smithi159.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-03-19T01:38:14.121 INFO:teuthology.orchestra.run.smithi092.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-03-19T01:38:14.162 INFO:teuthology.orchestra.run.smithi092.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-03-19T01:38:14.188 INFO:teuthology.orchestra.run.smithi092.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-03-19T01:38:14.221 INFO:teuthology.orchestra.run.smithi092.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-03-19T01:38:14.247 INFO:teuthology.orchestra.run.smithi092.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-03-19T01:38:14.281 INFO:teuthology.orchestra.run.smithi092.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-03-19T01:38:14.307 INFO:teuthology.orchestra.run.smithi092.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-03-19T01:38:14.339 INFO:teuthology.orchestra.run.smithi092.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-03-19T01:38:14.365 INFO:teuthology.orchestra.run.smithi092.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 522 kB/s | 31 kB 00:00 2024-03-19T01:38:14.407 INFO:teuthology.orchestra.run.smithi092.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-03-19T01:38:14.432 INFO:teuthology.orchestra.run.smithi092.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-03-19T01:38:14.466 INFO:teuthology.orchestra.run.smithi092.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-03-19T01:38:14.491 INFO:teuthology.orchestra.run.smithi092.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-03-19T01:38:14.492 INFO:teuthology.orchestra.run.smithi159.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 553 kB/s | 216 kB 00:00 2024-03-19T01:38:14.533 INFO:teuthology.orchestra.run.smithi092.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-03-19T01:38:14.558 INFO:teuthology.orchestra.run.smithi092.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-19T01:38:14.584 INFO:teuthology.orchestra.run.smithi159.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 381 kB/s | 35 kB 00:00 2024-03-19T01:38:14.617 INFO:teuthology.orchestra.run.smithi092.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-03-19T01:38:14.642 INFO:teuthology.orchestra.run.smithi092.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-03-19T01:38:14.676 INFO:teuthology.orchestra.run.smithi092.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-03-19T01:38:14.718 INFO:teuthology.orchestra.run.smithi159.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 334 kB/s | 44 kB 00:00 2024-03-19T01:38:14.801 INFO:teuthology.orchestra.run.smithi092.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-03-19T01:38:14.801 INFO:teuthology.orchestra.run.smithi159.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-03-19T01:38:14.935 INFO:teuthology.orchestra.run.smithi159.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-03-19T01:38:15.021 INFO:teuthology.orchestra.run.smithi092.stdout:(150/174): librados2-19.0.0-2206.gb6ae2491.el9. 9.8 MB/s | 3.4 MB 00:00 2024-03-19T01:38:15.096 INFO:teuthology.orchestra.run.smithi092.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-03-19T01:38:15.210 INFO:teuthology.orchestra.run.smithi159.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.9 MB/s | 4.1 MB 00:01 2024-03-19T01:38:15.655 INFO:teuthology.orchestra.run.smithi092.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 4.1 MB/s | 2.3 MB 00:00 2024-03-19T01:38:15.694 INFO:teuthology.orchestra.run.smithi159.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-03-19T01:38:15.855 INFO:teuthology.orchestra.run.smithi092.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 9.2 MB/s | 1.8 MB 00:00 2024-03-19T01:38:15.913 INFO:teuthology.orchestra.run.smithi092.stdout:(154/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-03-19T01:38:15.939 INFO:teuthology.orchestra.run.smithi092.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 505 kB/s | 42 kB 00:00 2024-03-19T01:38:15.997 INFO:teuthology.orchestra.run.smithi092.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-19T01:38:16.031 INFO:teuthology.orchestra.run.smithi092.stdout:(157/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-03-19T01:38:16.056 INFO:teuthology.orchestra.run.smithi092.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 623 kB/s | 36 kB 00:00 2024-03-19T01:38:16.115 INFO:teuthology.orchestra.run.smithi092.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-03-19T01:38:16.149 INFO:teuthology.orchestra.run.smithi092.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-03-19T01:38:16.174 INFO:teuthology.orchestra.run.smithi092.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 393 kB/s | 23 kB 00:00 2024-03-19T01:38:16.199 INFO:teuthology.orchestra.run.smithi092.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-19T01:38:16.233 INFO:teuthology.orchestra.run.smithi092.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-19T01:38:16.250 INFO:teuthology.orchestra.run.smithi092.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-19T01:38:16.292 INFO:teuthology.orchestra.run.smithi092.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-19T01:38:16.317 INFO:teuthology.orchestra.run.smithi092.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 428 kB/s | 29 kB 00:00 2024-03-19T01:38:16.450 INFO:teuthology.orchestra.run.smithi092.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 1.2 MB 00:00 2024-03-19T01:38:16.851 INFO:teuthology.orchestra.run.smithi092.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.7 MB/s | 2.3 MB 00:00 2024-03-19T01:38:16.951 INFO:teuthology.orchestra.run.smithi092.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.4 MB/s | 2.2 MB 00:00 2024-03-19T01:38:17.034 INFO:teuthology.orchestra.run.smithi092.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.5 MB/s | 465 kB 00:00 2024-03-19T01:38:17.060 INFO:teuthology.orchestra.run.smithi092.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 287 kB/s | 31 kB 00:00 2024-03-19T01:38:17.143 INFO:teuthology.orchestra.run.smithi092.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 891 kB/s | 74 kB 00:00 2024-03-19T01:38:17.243 INFO:teuthology.orchestra.run.smithi092.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.1 MB/s | 1.1 MB 00:00 2024-03-19T01:38:19.043 INFO:teuthology.orchestra.run.smithi092.stdout:(174/174): gperftools-libs-2.9.1-2.el9.x86_64.r 42 kB/s | 309 kB 00:07 2024-03-19T01:38:19.047 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-03-19T01:38:19.047 INFO:teuthology.orchestra.run.smithi092.stdout:Total 12 MB/s | 224 MB 00:18 2024-03-19T01:38:19.499 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-03-19T01:38:19.776 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-03-19T01:38:19.776 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-03-19T01:38:21.890 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-03-19T01:38:21.890 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-03-19T01:38:23.518 INFO:teuthology.orchestra.run.smithi159.stdout:(106/174): ceph-test-19.0.0-2206.gb6ae2491.el9. 2.8 MB/s | 48 MB 00:17 2024-03-19T01:38:28.800 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-03-19T01:38:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-19T01:38:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-19T01:38:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T01:38:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T01:38:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-19T01:38:29.098 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-19T01:38:29.132 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-19T01:38:29.157 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-19T01:38:29.210 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-19T01:38:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-19T01:38:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T01:38:29.687 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T01:38:29.726 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T01:38:29.801 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T01:38:29.838 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T01:38:29.886 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-19T01:38:30.056 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-19T01:38:30.112 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-19T01:38:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T01:38:30.235 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T01:38:30.257 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T01:38:30.306 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T01:38:30.345 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-19T01:38:30.359 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T01:38:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T01:38:30.543 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T01:38:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T01:38:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-19T01:38:30.769 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-19T01:38:30.916 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T01:38:30.959 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T01:38:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T01:38:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T01:38:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T01:38:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-19T01:38:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-19T01:38:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-19T01:38:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-19T01:38:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-19T01:38:31.704 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-19T01:38:31.758 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-19T01:38:31.804 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-19T01:38:31.868 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-19T01:38:31.903 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-19T01:38:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-19T01:38:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-19T01:38:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-19T01:38:32.076 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-19T01:38:32.112 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-19T01:38:32.155 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-19T01:38:32.184 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-19T01:38:32.253 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-19T01:38:32.307 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-19T01:38:32.380 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-19T01:38:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-19T01:38:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-19T01:38:33.085 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-19T01:38:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-19T01:38:33.265 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-19T01:38:33.308 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T01:38:33.316 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T01:38:33.359 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-19T01:38:33.405 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-19T01:38:33.443 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-19T01:38:33.474 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-19T01:38:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-19T01:38:33.945 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-19T01:38:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-19T01:38:34.060 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-19T01:38:34.199 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-19T01:38:34.247 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-19T01:38:34.301 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-19T01:38:34.457 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-19T01:38:34.489 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-19T01:38:34.526 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-19T01:38:34.573 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-19T01:38:34.609 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-19T01:38:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-19T01:38:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-19T01:38:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-19T01:38:34.882 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-19T01:38:34.912 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-19T01:38:34.954 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-19T01:38:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-19T01:38:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-19T01:38:35.137 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-19T01:38:35.162 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-19T01:38:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-19T01:38:35.265 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-19T01:38:35.393 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-19T01:38:35.492 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-19T01:38:35.534 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-19T01:38:35.583 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-19T01:38:35.647 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-19T01:38:35.831 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-19T01:38:35.897 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-19T01:38:36.025 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-19T01:38:36.062 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-19T01:38:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-19T01:38:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-19T01:38:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-19T01:38:36.187 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-19T01:38:36.226 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-19T01:38:36.249 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-19T01:38:36.279 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-19T01:38:36.354 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-19T01:38:36.391 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-19T01:38:36.423 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-19T01:38:36.464 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-19T01:38:36.499 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-19T01:38:36.587 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-19T01:38:36.669 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-19T01:38:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-19T01:38:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-19T01:38:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-19T01:38:36.922 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-19T01:38:36.971 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-19T01:38:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-19T01:38:37.743 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-19T01:38:37.777 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-19T01:38:37.825 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-19T01:38:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-19T01:38:37.961 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-19T01:38:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-19T01:38:38.159 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-19T01:38:38.237 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-19T01:38:38.878 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-19T01:38:38.997 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-19T01:38:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-19T01:38:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-19T01:38:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-19T01:38:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-19T01:38:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-19T01:38:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-19T01:38:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T01:38:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T01:38:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-19T01:38:40.869 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-19T01:38:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-19T01:38:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T01:38:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T01:38:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-19T01:38:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T01:38:43.711 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T01:38:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T01:38:44.306 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T01:38:44.399 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T01:38:44.399 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-19T01:38:44.399 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-19T01:38:44.400 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:38:44.432 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T01:38:56.184 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T01:38:56.184 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-03-19T01:38:56.184 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-03-19T01:38:56.184 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-03-19T01:38:56.184 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-03-19T01:38:56.184 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-03-19T01:38:56.184 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-03-19T01:38:56.184 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-03-19T01:38:56.185 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:38:56.336 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T01:38:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T01:38:56.350 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:38:56.350 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-19T01:38:56.350 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:38:57.822 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T01:38:57.847 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T01:38:57.847 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:38:57.847 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-19T01:38:57.847 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T01:38:57.847 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T01:38:57.847 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:38:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T01:38:58.095 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T01:38:58.095 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:38:58.095 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-19T01:38:58.095 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T01:38:58.095 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T01:38:58.095 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:38:58.531 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T01:38:58.554 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T01:38:58.554 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:38:58.554 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-19T01:38:58.554 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T01:38:58.554 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T01:38:58.554 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:38:58.658 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-19T01:38:59.066 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T01:38:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T01:38:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-19T01:38:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-19T01:38:59.204 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T01:38:59.436 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T01:38:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T01:39:00.428 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T01:39:00.455 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T01:39:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T01:39:00.717 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-19T01:39:00.741 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T01:39:00.763 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T01:39:00.763 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:39:00.763 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-19T01:39:00.763 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T01:39:00.763 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T01:39:00.764 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:39:00.808 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T01:39:00.819 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T01:39:00.866 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-19T01:39:01.735 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-19T01:39:01.768 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T01:39:01.791 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T01:39:01.791 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:39:01.791 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-19T01:39:01.791 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T01:39:01.791 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T01:39:01.791 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:39:01.836 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T01:39:01.857 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T01:39:01.857 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:39:01.857 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-19T01:39:01.858 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:39:01.911 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T01:39:01.933 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T01:39:01.933 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:39:01.933 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-19T01:39:01.933 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T01:39:01.933 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T01:39:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:39:02.237 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T01:39:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T01:39:02.259 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:39:02.259 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-19T01:39:02.259 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T01:39:02.259 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T01:39:02.259 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:39:06.716 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T01:39:06.757 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-19T01:39:06.787 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-19T01:39:06.815 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-19T01:39:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-19T01:39:07.115 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-19T01:39:07.162 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-19T01:39:07.209 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-19T01:39:07.262 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-19T01:39:07.297 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-19T01:39:07.330 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-19T01:39:07.370 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-19T01:39:07.400 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-19T01:39:07.484 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-19T01:39:07.559 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-19T01:39:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-19T01:39:08.154 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T01:39:08.189 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T01:39:08.209 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T01:39:09.021 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-19T01:39:09.101 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-19T01:39:09.142 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-19T01:39:09.234 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-19T01:39:09.263 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-19T01:39:09.288 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-19T01:39:09.313 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-19T01:39:09.340 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-19T01:39:09.340 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T01:39:09.427 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T01:39:09.457 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-19T01:39:09.457 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T01:39:09.525 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T01:39:09.552 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-19T01:39:09.585 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-19T01:39:09.610 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-19T01:39:09.635 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-19T01:39:09.669 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-19T01:39:09.716 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-19T01:39:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-19T01:39:09.773 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-19T01:39:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-19T01:39:09.823 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-19T01:39:09.848 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-19T01:39:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T01:39:09.855 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T01:39:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T01:39:10.035 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T01:39:10.126 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T01:39:10.159 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-19T01:39:10.185 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-19T01:39:10.217 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-19T01:39:10.244 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-19T01:39:10.309 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T01:39:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T01:39:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-19T01:39:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-19T01:39:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-19T01:39:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-19T01:39:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-19T01:39:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-19T01:39:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-19T01:39:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-19T01:39:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-19T01:39:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-19T01:39:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-19T01:39:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-19T01:39:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-19T01:39:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-19T01:39:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-19T01:39:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-19T01:39:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-19T01:39:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-19T01:39:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: acl-2.3.1-4.el9.x86_64 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-19T01:39:18.059 INFO:teuthology.orchestra.run.smithi092.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-19T01:39:18.060 INFO:teuthology.orchestra.run.smithi092.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.061 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.062 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-19T01:39:18.063 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: jq-1.6-16.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-19T01:39:18.064 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-19T01:39:18.065 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.11-1.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-19T01:39:18.066 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-19T01:39:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-19T01:39:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-19T01:39:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-19T01:39:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-19T01:39:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-19T01:39:18.068 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-19T01:39:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-19T01:39:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-19T01:39:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-19T01:39:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-19T01:39:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-19T01:39:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-19T01:39:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-19T01:39:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-19T01:39:18.069 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-19T01:39:18.070 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-19T01:39:18.071 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.072 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:39:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2024-03-19T01:39:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-19T01:39:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-19T01:39:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-19T01:39:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-19T01:39:18.073 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-19T01:39:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-19T01:39:18.074 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:39:18.074 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-03-19T01:39:18.554 DEBUG:teuthology.parallel:result is None 2024-03-19T01:40:25.316 INFO:teuthology.orchestra.run.smithi159.stdout:[MIRROR] abseil-cpp-20211102.0-3.el9.x86_64.rpm: Curl error (28): Timeout was reached for https://ftp-nyc.osuosl.org/pub/fedora-epel/9/Everything/x86_64/Packages/a/abseil-cpp-20211102.0-3.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: Connection timed out] 2024-03-19T01:40:25.317 INFO:teuthology.orchestra.run.smithi159.stdout:[MIRROR] gperftools-libs-2.9.1-2.el9.x86_64.rpm: Curl error (28): Timeout was reached for https://ftp-nyc.osuosl.org/pub/fedora-epel/9/Everything/x86_64/Packages/g/gperftools-libs-2.9.1-2.el9.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: Connection timed out] 2024-03-19T01:40:25.893 INFO:teuthology.orchestra.run.smithi159.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 kB/s | 309 kB 02:10 2024-03-19T01:40:25.926 INFO:teuthology.orchestra.run.smithi159.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 4.2 kB/s | 548 kB 02:10 2024-03-19T01:40:26.018 INFO:teuthology.orchestra.run.smithi159.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-03-19T01:40:26.094 INFO:teuthology.orchestra.run.smithi159.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 650 kB/s | 49 kB 00:00 2024-03-19T01:40:26.202 INFO:teuthology.orchestra.run.smithi159.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-03-19T01:40:26.328 INFO:teuthology.orchestra.run.smithi159.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-03-19T01:40:26.420 INFO:teuthology.orchestra.run.smithi159.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-03-19T01:40:26.620 INFO:teuthology.orchestra.run.smithi159.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-03-19T01:40:26.645 INFO:teuthology.orchestra.run.smithi159.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 838 kB 00:00 2024-03-19T01:40:26.704 INFO:teuthology.orchestra.run.smithi159.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-03-19T01:40:26.737 INFO:teuthology.orchestra.run.smithi159.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2024-03-19T01:40:26.780 INFO:teuthology.orchestra.run.smithi159.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-03-19T01:40:26.821 INFO:teuthology.orchestra.run.smithi159.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-03-19T01:40:26.863 INFO:teuthology.orchestra.run.smithi159.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-19T01:40:26.922 INFO:teuthology.orchestra.run.smithi159.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-03-19T01:40:26.947 INFO:teuthology.orchestra.run.smithi159.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-03-19T01:40:27.014 INFO:teuthology.orchestra.run.smithi159.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-19T01:40:27.106 INFO:teuthology.orchestra.run.smithi159.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-03-19T01:40:27.131 INFO:teuthology.orchestra.run.smithi159.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-19T01:40:27.190 INFO:teuthology.orchestra.run.smithi159.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-03-19T01:40:27.215 INFO:teuthology.orchestra.run.smithi159.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-03-19T01:40:27.257 INFO:teuthology.orchestra.run.smithi159.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-03-19T01:40:27.282 INFO:teuthology.orchestra.run.smithi159.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-03-19T01:40:27.324 INFO:teuthology.orchestra.run.smithi159.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-03-19T01:40:27.433 INFO:teuthology.orchestra.run.smithi159.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-03-19T01:40:27.467 INFO:teuthology.orchestra.run.smithi159.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-03-19T01:40:27.517 INFO:teuthology.orchestra.run.smithi159.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-03-19T01:40:27.542 INFO:teuthology.orchestra.run.smithi159.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 769 kB/s | 58 kB 00:00 2024-03-19T01:40:27.601 INFO:teuthology.orchestra.run.smithi159.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-19T01:40:27.626 INFO:teuthology.orchestra.run.smithi159.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-19T01:40:27.677 INFO:teuthology.orchestra.run.smithi159.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-19T01:40:27.718 INFO:teuthology.orchestra.run.smithi159.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-03-19T01:40:27.744 INFO:teuthology.orchestra.run.smithi159.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 461 kB/s | 31 kB 00:00 2024-03-19T01:40:27.794 INFO:teuthology.orchestra.run.smithi159.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-19T01:40:27.819 INFO:teuthology.orchestra.run.smithi159.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-19T01:40:27.854 INFO:teuthology.orchestra.run.smithi159.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 608 kB/s | 36 kB 00:00 2024-03-19T01:40:27.887 INFO:teuthology.orchestra.run.smithi159.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 758 kB/s | 51 kB 00:00 2024-03-19T01:40:27.937 INFO:teuthology.orchestra.run.smithi159.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-19T01:40:27.962 INFO:teuthology.orchestra.run.smithi159.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-19T01:40:28.029 INFO:teuthology.orchestra.run.smithi159.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-03-19T01:40:28.054 INFO:teuthology.orchestra.run.smithi159.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-03-19T01:40:28.105 INFO:teuthology.orchestra.run.smithi159.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-19T01:40:28.172 INFO:teuthology.orchestra.run.smithi159.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-03-19T01:40:28.588 INFO:teuthology.orchestra.run.smithi159.stdout:(150/174): librbd1-19.0.0-2206.gb6ae2491.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-03-19T01:40:28.714 INFO:teuthology.orchestra.run.smithi159.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 321 kB 00:00 2024-03-19T01:40:29.164 INFO:teuthology.orchestra.run.smithi159.stdout:(152/174): librados2-19.0.0-2206.gb6ae2491.el9. 3.2 MB/s | 3.4 MB 00:01 2024-03-19T01:40:29.581 INFO:teuthology.orchestra.run.smithi159.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.7 MB/s | 2.3 MB 00:00 2024-03-19T01:40:29.690 INFO:teuthology.orchestra.run.smithi159.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 388 kB/s | 42 kB 00:00 2024-03-19T01:40:29.798 INFO:teuthology.orchestra.run.smithi159.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-03-19T01:40:29.907 INFO:teuthology.orchestra.run.smithi159.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 658 kB/s | 71 kB 00:00 2024-03-19T01:40:30.016 INFO:teuthology.orchestra.run.smithi159.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 336 kB/s | 36 kB 00:00 2024-03-19T01:40:30.057 INFO:teuthology.orchestra.run.smithi159.stdout:(158/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.1 MB/s | 1.8 MB 00:00 2024-03-19T01:40:30.124 INFO:teuthology.orchestra.run.smithi159.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-03-19T01:40:30.166 INFO:teuthology.orchestra.run.smithi159.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-19T01:40:30.225 INFO:teuthology.orchestra.run.smithi159.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2024-03-19T01:40:30.275 INFO:teuthology.orchestra.run.smithi159.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 108 kB 00:00 2024-03-19T01:40:30.334 INFO:teuthology.orchestra.run.smithi159.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-19T01:40:30.384 INFO:teuthology.orchestra.run.smithi159.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-19T01:40:30.443 INFO:teuthology.orchestra.run.smithi159.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 589 kB/s | 64 kB 00:00 2024-03-19T01:40:30.485 INFO:teuthology.orchestra.run.smithi159.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 286 kB/s | 29 kB 00:00 2024-03-19T01:40:30.568 INFO:teuthology.orchestra.run.smithi159.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-03-19T01:40:31.585 INFO:teuthology.orchestra.run.smithi159.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.0 MB/s | 2.2 MB 00:01 2024-03-19T01:40:31.777 INFO:teuthology.orchestra.run.smithi159.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 1.9 MB/s | 2.3 MB 00:01 2024-03-19T01:40:31.877 INFO:teuthology.orchestra.run.smithi159.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 311 kB/s | 31 kB 00:00 2024-03-19T01:40:32.069 INFO:teuthology.orchestra.run.smithi159.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 961 kB/s | 465 kB 00:00 2024-03-19T01:40:32.186 INFO:teuthology.orchestra.run.smithi159.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 637 kB/s | 74 kB 00:00 2024-03-19T01:40:33.174 INFO:teuthology.orchestra.run.smithi159.stdout:[MIRROR] grpc-data-1.46.7-10.el9.noarch.rpm: Curl error (28): Timeout was reached for https://ftp-nyc.osuosl.org/pub/fedora-epel/9/Everything/x86_64/Packages/g/grpc-data-1.46.7-10.el9.noarch.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: Connection timed out] 2024-03-19T01:40:33.528 INFO:teuthology.orchestra.run.smithi159.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 666 kB/s | 1.1 MB 00:01 2024-03-19T01:40:33.553 INFO:teuthology.orchestra.run.smithi159.stdout:(174/174): grpc-data-1.46.7-10.el9.noarch.rpm 153 B/s | 19 kB 02:10 2024-03-19T01:40:33.556 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-03-19T01:40:33.557 INFO:teuthology.orchestra.run.smithi159.stdout:Total 1.5 MB/s | 224 MB 02:31 2024-03-19T01:40:34.022 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-03-19T01:40:34.294 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-03-19T01:40:34.294 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-03-19T01:40:35.999 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-03-19T01:40:35.999 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-03-19T01:40:42.930 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-03-19T01:40:43.107 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-19T01:40:43.150 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-19T01:40:43.188 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T01:40:43.197 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-19T01:40:43.246 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-19T01:40:43.308 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-19T01:40:43.342 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-19T01:40:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-19T01:40:43.418 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-19T01:40:43.738 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-19T01:40:43.794 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T01:40:43.915 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T01:40:43.953 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T01:40:44.044 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T01:40:44.074 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T01:40:44.120 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-19T01:40:44.298 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-19T01:40:44.355 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-19T01:40:44.383 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T01:40:44.488 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T01:40:44.519 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T01:40:44.592 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 17/198 2024-03-19T01:40:44.640 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-19T01:40:44.662 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T01:40:44.678 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-19T01:40:44.871 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T01:40:44.910 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-19T01:40:44.951 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-19T01:40:45.053 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-19T01:40:45.152 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T01:40:45.216 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-19T01:40:45.284 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T01:40:45.356 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T01:40:45.569 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-19T01:40:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-19T01:40:45.771 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-19T01:40:45.810 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-19T01:40:45.846 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-19T01:40:45.881 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-19T01:40:45.955 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-19T01:40:46.009 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-19T01:40:46.056 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-19T01:40:46.128 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-19T01:40:46.164 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-19T01:40:46.207 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-19T01:40:46.240 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-19T01:40:46.268 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-19T01:40:46.336 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-19T01:40:46.372 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 39/198 2024-03-19T01:40:46.416 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 40/198 2024-03-19T01:40:46.453 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 41/198 2024-03-19T01:40:46.521 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-19T01:40:46.576 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-19T01:40:46.649 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-19T01:40:46.782 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-19T01:40:46.837 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-19T01:40:47.354 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-19T01:40:47.440 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-19T01:40:47.523 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-19T01:40:47.594 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T01:40:47.601 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-19T01:40:47.634 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-19T01:40:47.675 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-19T01:40:47.721 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 53/198 2024-03-19T01:40:47.759 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-19T01:40:47.981 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-19T01:40:48.146 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-19T01:40:48.228 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-19T01:40:48.272 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-19T01:40:48.402 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-19T01:40:48.442 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-19T01:40:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-19T01:40:48.584 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-19T01:40:48.818 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-19T01:40:48.988 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-19T01:40:49.035 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-19T01:40:49.070 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-19T01:40:49.113 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-19T01:40:49.155 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-19T01:40:49.287 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-19T01:40:49.337 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-19T01:40:49.366 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-19T01:40:49.407 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-19T01:40:49.501 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-19T01:40:49.541 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-19T01:40:49.581 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-19T01:40:49.607 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-19T01:40:49.631 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-19T01:40:49.703 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-19T01:40:49.823 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-19T01:40:49.930 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-19T01:40:49.973 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-19T01:40:50.022 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-19T01:40:50.077 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-19T01:40:50.268 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-19T01:40:50.319 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-19T01:40:50.447 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-19T01:40:50.484 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-19T01:40:50.516 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-19T01:40:50.547 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-19T01:40:50.575 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-19T01:40:50.626 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-19T01:40:50.673 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-19T01:40:50.713 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-19T01:40:50.743 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-19T01:40:50.819 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-19T01:40:50.854 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-19T01:40:50.887 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-19T01:40:50.928 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-19T01:40:50.963 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-19T01:40:51.052 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-19T01:40:51.134 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-19T01:40:51.185 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-19T01:40:51.217 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-19T01:40:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-19T01:40:51.361 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-19T01:40:51.410 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-19T01:40:52.136 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-19T01:40:52.183 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-19T01:40:52.216 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-19T01:40:52.264 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-19T01:40:52.344 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-19T01:40:52.376 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-19T01:40:52.553 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-19T01:40:52.582 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-19T01:40:52.659 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-19T01:40:53.297 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-19T01:40:53.412 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-19T01:40:54.615 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-19T01:40:54.708 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-19T01:40:54.747 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-19T01:40:54.787 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-19T01:40:54.822 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-19T01:40:55.135 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-19T01:40:55.169 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T01:40:55.242 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 124/198 2024-03-19T01:40:55.274 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 125/198 2024-03-19T01:40:55.310 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-19T01:40:55.771 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-19T01:40:55.805 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T01:40:55.876 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 128/198 2024-03-19T01:40:55.899 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 129/198 2024-03-19T01:40:58.161 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T01:40:58.197 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T01:40:58.736 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 130/198 2024-03-19T01:40:58.775 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T01:40:58.856 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 131/198 2024-03-19T01:40:58.856 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-19T01:40:58.857 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-19T01:40:58.857 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:40:58.885 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T01:41:10.622 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 132/198 2024-03-19T01:41:10.622 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2024-03-19T01:41:10.622 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2024-03-19T01:41:10.622 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2024-03-19T01:41:10.622 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2024-03-19T01:41:10.623 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2024-03-19T01:41:10.623 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /root 2024-03-19T01:41:10.623 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2024-03-19T01:41:10.623 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:10.799 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T01:41:10.814 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 133/198 2024-03-19T01:41:10.814 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:41:10.814 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-19T01:41:10.814 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:12.294 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T01:41:12.319 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 134/198 2024-03-19T01:41:12.319 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:41:12.319 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-19T01:41:12.319 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T01:41:12.319 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-19T01:41:12.320 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:12.561 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T01:41:12.583 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 135/198 2024-03-19T01:41:12.583 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:41:12.583 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-19T01:41:12.584 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T01:41:12.584 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-19T01:41:12.584 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:13.029 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T01:41:13.051 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 136/198 2024-03-19T01:41:13.051 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:41:13.051 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-19T01:41:13.051 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T01:41:13.051 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-19T01:41:13.051 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:13.139 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-19T01:41:13.572 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T01:41:13.612 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 138/198 2024-03-19T01:41:13.644 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 139/198 2024-03-19T01:41:13.704 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 140/198 2024-03-19T01:41:13.727 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T01:41:13.961 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 141/198 2024-03-19T01:41:13.994 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T01:41:14.957 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 142/198 2024-03-19T01:41:14.986 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T01:41:15.087 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 143/198 2024-03-19T01:41:15.251 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 144/198 2024-03-19T01:41:15.289 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T01:41:15.312 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 145/198 2024-03-19T01:41:15.312 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:41:15.312 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-19T01:41:15.312 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T01:41:15.312 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-19T01:41:15.313 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:15.365 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T01:41:15.375 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 146/198 2024-03-19T01:41:15.421 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 147/198 2024-03-19T01:41:16.315 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-19T01:41:16.341 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T01:41:16.364 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 149/198 2024-03-19T01:41:16.364 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:41:16.364 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-19T01:41:16.364 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T01:41:16.364 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-19T01:41:16.364 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:16.409 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T01:41:16.431 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 150/198 2024-03-19T01:41:16.431 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:41:16.431 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-19T01:41:16.431 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:16.476 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T01:41:16.498 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T01:41:16.498 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:41:16.498 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-19T01:41:16.498 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T01:41:16.498 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-19T01:41:16.498 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:16.802 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T01:41:16.824 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 152/198 2024-03-19T01:41:16.824 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-19T01:41:16.824 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-19T01:41:16.824 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T01:41:16.824 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-19T01:41:16.824 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:21.281 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T01:41:21.331 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 154/198 2024-03-19T01:41:21.369 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 155/198 2024-03-19T01:41:21.397 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-19T01:41:21.576 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 157/198 2024-03-19T01:41:21.688 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-19T01:41:21.740 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 159/198 2024-03-19T01:41:21.777 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-19T01:41:21.829 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-19T01:41:21.872 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-19T01:41:21.896 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 163/198 2024-03-19T01:41:21.937 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-19T01:41:21.967 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-19T01:41:22.029 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-19T01:41:22.090 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-19T01:41:22.124 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-19T01:41:22.752 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T01:41:22.796 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T01:41:22.816 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-19T01:41:23.634 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 170/198 2024-03-19T01:41:23.734 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-19T01:41:23.774 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-19T01:41:23.855 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-19T01:41:23.896 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-19T01:41:23.929 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-19T01:41:23.963 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-19T01:41:23.989 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-19T01:41:23.989 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T01:41:24.069 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-19T01:41:24.098 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-19T01:41:24.098 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T01:41:24.183 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-19T01:41:24.209 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-19T01:41:24.243 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-19T01:41:24.268 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-19T01:41:24.293 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-19T01:41:24.326 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-19T01:41:24.365 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-19T01:41:24.398 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-19T01:41:24.431 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-19T01:41:24.457 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-19T01:41:24.482 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-19T01:41:24.508 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-19T01:41:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T01:41:24.514 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T01:41:24.694 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-19T01:41:24.694 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T01:41:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-19T01:41:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-19T01:41:24.842 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-19T01:41:24.869 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-19T01:41:24.894 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-19T01:41:24.959 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T01:41:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-19T01:41:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 1/198 2024-03-19T01:41:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2/198 2024-03-19T01:41:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 3/198 2024-03-19T01:41:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 4/198 2024-03-19T01:41:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2 5/198 2024-03-19T01:41:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 6/198 2024-03-19T01:41:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 7/198 2024-03-19T01:41:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 8/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 9/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 10/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 11/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 12/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 13/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 14/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_ 15/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 16/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 17/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_6 18/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86 19/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 20/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el 21/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9. 22/198 2024-03-19T01:41:29.748 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_6 23/198 2024-03-19T01:41:29.749 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 24/198 2024-03-19T01:41:29.749 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 25/198 2024-03-19T01:41:29.749 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 26/198 2024-03-19T01:41:29.749 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 27/198 2024-03-19T01:41:29.749 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 28/198 2024-03-19T01:41:29.749 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 29/198 2024-03-19T01:41:29.769 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491. 30/198 2024-03-19T01:41:29.769 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noa 31/198 2024-03-19T01:41:29.769 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.n 32/198 2024-03-19T01:41:29.769 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6a 33/198 2024-03-19T01:41:29.769 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el 34/198 2024-03-19T01:41:29.769 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 35/198 2024-03-19T01:41:29.769 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.e 36/198 2024-03-19T01:41:29.769 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 37/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 38/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 39/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-19T01:41:29.770 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-19T01:41:29.771 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-19T01:41:29.772 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-19T01:41:29.773 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-19T01:41:29.774 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-19T01:41:29.775 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-19T01:41:29.776 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-19T01:41:29.777 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 151/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 153/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-19T01:41:29.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-19T01:41:29.779 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-19T01:41:29.780 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-19T01:41:29.781 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-19T01:41:29.781 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-19T01:41:29.781 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-19T01:41:29.781 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-19T01:41:29.781 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-19T01:41:29.781 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-19T01:41:32.794 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-19T01:41:32.794 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout: acl-2.3.1-4.el9.x86_64 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-19T01:41:32.795 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-19T01:41:32.796 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.2-8.el9.noarch 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.7.4-5.el9.x86_64 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.797 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.798 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:19.0.0-2206.gb6ae2491.el9.noarch 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.4-9.el9.x86_64 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-19T01:41:32.799 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: jq-1.6-16.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-19T01:41:32.800 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: lua-5.4.4-4.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-19T01:41:32.801 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-13.11-1.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-19T01:41:32.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-19T01:41:32.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-19T01:41:32.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply-3.11-14.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-19T01:41:32.805 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-19T01:41:32.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.0.0-2206.gb6ae2491.el9.x86_64 2024-03-19T01:41:32.807 INFO:teuthology.orchestra.run.smithi159.stdout: re2-1:20211101-3.el9.x86_64 2024-03-19T01:41:32.808 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-19T01:41:32.808 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-19T01:41:32.808 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-19T01:41:32.808 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-19T01:41:32.808 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-19T01:41:32.808 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-19T01:41:32.808 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:41:32.808 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-03-19T01:41:33.240 DEBUG:teuthology.parallel:result is None 2024-03-19T01:41:33.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:41:33.460 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-19T01:41:33.487 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-19T01:41:33.487 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-19T01:41:33.487 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-19T01:41:33.489 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:41:33.625 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-19T01:41:33.652 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-19T01:41:33.652 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-19T01:41:33.653 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-19T01:41:33.654 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:41:33.867 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-19T01:41:33.889 INFO:teuthology.orchestra.run.smithi159.stdout:19.0.0-2206.gb6ae2491.el9 2024-03-19T01:41:33.889 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2206.gb6ae2491.el9 2024-03-19T01:41:33.889 INFO:teuthology.task.install:The correct ceph version 19.0.0-2206.gb6ae2491 is installed. 2024-03-19T01:41:33.891 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-19T01:41:33.891 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:41:33.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-19T01:41:33.925 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:41:33.925 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-19T01:41:33.960 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:41:33.960 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-19T01:41:33.990 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-19T01:41:33.990 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:41:33.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-03-19T01:41:34.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-19T01:41:34.091 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:41:34.091 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-03-19T01:41:34.122 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-19T01:41:34.192 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:41:34.192 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2024-03-19T01:41:34.219 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-19T01:41:34.285 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-19T01:41:34.285 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:41:34.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-19T01:41:34.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-19T01:41:34.402 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:41:34.402 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-19T01:41:34.433 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-19T01:41:34.504 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:41:34.504 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-19T01:41:34.531 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-19T01:41:34.598 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-19T01:41:34.599 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:41:34.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-03-19T01:41:34.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-19T01:41:34.693 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:41:34.693 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-03-19T01:41:34.724 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-19T01:41:34.795 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:41:34.795 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2024-03-19T01:41:34.821 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-19T01:41:34.888 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-19T01:41:34.974 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-19T01:41:34.974 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:41:34.974 INFO:tasks.cephadm:Cluster fsid is d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:41:34.974 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-19T01:41:34.975 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.92', 'mon.b': '172.21.15.144', 'mon.c': '172.21.15.159'} 2024-03-19T01:41:34.975 INFO:tasks.cephadm:First mon is mon.a on smithi092 2024-03-19T01:41:34.975 INFO:tasks.cephadm:First mgr is x 2024-03-19T01:41:34.975 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-19T01:41:34.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-03-19T01:41:35.002 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-03-19T01:41:35.034 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2024-03-19T01:41:35.061 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-19T01:41:35.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:41:35.202 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b6ae24918b03b1b8a19c9263a50655bba80bb810', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77736/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2206-gb6ae2491', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2206.gb6ae2491'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-18 20:42:27.872142', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240318.181317', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-19T01:41:35.341 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-batrick-testing-20240318.181317, sha1 b6ae24918b03b1b8a19c9263a50655bba80bb810 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:41:35.342 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm 2024-03-19T01:41:35.343 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-19T01:41:35.927 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 19 01:41 /home/ubuntu/cephtest/cephadm 2024-03-19T01:41:35.927 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-19T01:41:36.384 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 19 01:41 /home/ubuntu/cephtest/cephadm 2024-03-19T01:41:36.385 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240318.181317/b6ae24918b03b1b8a19c9263a50655bba80bb810/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-19T01:41:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-r--r--. 1 ubuntu ubuntu 767404 Mar 19 01:41 /home/ubuntu/cephtest/cephadm 2024-03-19T01:41:36.981 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-19T01:41:36.999 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-19T01:41:37.018 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-19T01:41:37.071 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 on all hosts... 2024-03-19T01:41:37.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-19T01:41:37.073 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-19T01:41:37.076 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 pull 2024-03-19T01:41:37.852 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T01:41:37.965 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T01:41:37.966 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T01:42:10.286 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-19T01:42:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T01:42:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-19T01:42:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-03-19T01:42:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-19T01:42:10.287 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-03-19T01:42:10.287 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-19T01:42:25.177 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:42:25.177 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T01:42:25.178 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-19T01:42:25.178 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-03-19T01:42:25.178 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-19T01:42:25.178 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T01:42:25.178 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:42:26.381 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-19T01:42:26.381 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev)", 2024-03-19T01:42:26.381 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c", 2024-03-19T01:42:26.381 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2024-03-19T01:42:26.381 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639" 2024-03-19T01:42:26.381 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-03-19T01:42:26.381 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-19T01:42:26.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-03-19T01:42:26.438 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-03-19T01:42:26.472 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2024-03-19T01:42:26.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-03-19T01:42:26.531 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-03-19T01:42:26.561 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2024-03-19T01:42:26.588 INFO:tasks.cephadm:Writing seed config... 2024-03-19T01:42:26.589 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-19T01:42:26.589 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-19T01:42:26.589 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-19T01:42:26.589 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-19T01:42:26.589 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-19T01:42:26.589 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-19T01:42:26.590 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-19T01:42:26.591 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-19T01:42:26.592 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-19T01:42:26.592 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-19T01:42:26.592 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-19T01:42:26.592 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-19T01:42:26.592 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-19T01:42:26.592 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-19T01:42:26.592 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-19T01:42:26.592 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-19T01:42:26.593 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:42:26.593 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-19T01:42:26.609 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d1e857f4-e591-11ee-95c9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-19T01:42:26.609 DEBUG:teuthology.orchestra.run.smithi092:mon.a> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.a.service 2024-03-19T01:42:26.651 DEBUG:teuthology.orchestra.run.smithi092:mgr.x> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.x.service 2024-03-19T01:42:26.694 INFO:tasks.cephadm:Bootstrapping... 2024-03-19T01:42:26.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 -v bootstrap --fsid d1e857f4-e591-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.92 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-19T01:42:26.889 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-03-19T01:42:26.889 INFO:teuthology.orchestra.run.smithi092.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810', '-v', 'bootstrap', '--fsid', 'd1e857f4-e591-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.92', '--skip-admin-label'] 2024-03-19T01:42:26.913 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-19T01:42:26.913 INFO:teuthology.orchestra.run.smithi092.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-19T01:42:26.913 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying podman|docker is present... 2024-03-19T01:42:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-19T01:42:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying lvm2 is present... 2024-03-19T01:42:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying time synchronization is in place... 2024-03-19T01:42:26.945 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-19T01:42:26.945 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-19T01:42:26.951 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-19T01:42:26.951 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-03-19T01:42:26.957 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-03-19T01:42:26.963 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-03-19T01:42:26.963 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-03-19T01:42:26.964 INFO:teuthology.orchestra.run.smithi092.stdout:Repeating the final host check... 2024-03-19T01:42:26.985 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-19T01:42:26.985 INFO:teuthology.orchestra.run.smithi092.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-19T01:42:26.985 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl is present 2024-03-19T01:42:26.985 INFO:teuthology.orchestra.run.smithi092.stdout:lvcreate is present 2024-03-19T01:42:26.992 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-19T01:42:26.992 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-19T01:42:26.998 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-19T01:42:26.998 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-03-19T01:42:27.004 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout enabled 2024-03-19T01:42:27.010 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout active 2024-03-19T01:42:27.010 INFO:teuthology.orchestra.run.smithi092.stdout:Unit chronyd.service is enabled and running 2024-03-19T01:42:27.010 INFO:teuthology.orchestra.run.smithi092.stdout:Host looks OK 2024-03-19T01:42:27.011 INFO:teuthology.orchestra.run.smithi092.stdout:Cluster fsid: d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:42:27.011 INFO:teuthology.orchestra.run.smithi092.stdout:Acquiring lock 139980677103520 on /run/cephadm/d1e857f4-e591-11ee-95c9-87774f69a715.lock 2024-03-19T01:42:27.011 INFO:teuthology.orchestra.run.smithi092.stdout:Lock 139980677103520 acquired on /run/cephadm/d1e857f4-e591-11ee-95c9-87774f69a715.lock 2024-03-19T01:42:27.011 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 3300 ... 2024-03-19T01:42:27.011 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying IP 172.21.15.92 port 6789 ... 2024-03-19T01:42:27.011 INFO:teuthology.orchestra.run.smithi092.stdout:Base mon IP(s) is [172.21.15.92:3300, 172.21.15.92:6789], mon addrv is [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-03-19T01:42:27.014 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.92 metric 100 2024-03-19T01:42:27.015 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.92 metric 100 2024-03-19T01:42:27.017 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-19T01:42:27.017 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-19T01:42:27.017 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1747sec hoplimit 64 pref medium 2024-03-19T01:42:27.019 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-19T01:42:27.019 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-19T01:42:27.019 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-19T01:42:27.019 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-19T01:42:27.020 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2024-03-19T01:42:27.020 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-19T01:42:27.020 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-03-19T01:42:27.020 INFO:teuthology.orchestra.run.smithi092.stdout:Mon IP `172.21.15.92` is in CIDR network `172.21.0.0/20` 2024-03-19T01:42:27.020 INFO:teuthology.orchestra.run.smithi092.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-19T01:42:27.021 INFO:teuthology.orchestra.run.smithi092.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-19T01:42:27.021 INFO:teuthology.orchestra.run.smithi092.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T01:42:28.106 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stdout 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-19T01:42:28.106 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810... 2024-03-19T01:42:28.107 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Getting image source signatures 2024-03-19T01:42:28.107 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-19T01:42:28.107 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying blob sha256:93b5a8d362aeee7e8b1f0bbf18238e1bb256c50573ad5b72aff9e382b1bb3ccb 2024-03-19T01:42:28.107 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Copying config sha256:2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c 2024-03-19T01:42:28.107 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-19T01:42:28.979 INFO:teuthology.orchestra.run.smithi092.stdout:ceph: stdout ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-19T01:42:28.979 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph version: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev) 2024-03-19T01:42:28.979 INFO:teuthology.orchestra.run.smithi092.stdout:Extracting ceph user uid/gid from container image... 2024-03-19T01:42:29.909 INFO:teuthology.orchestra.run.smithi092.stdout:stat: stdout 167 167 2024-03-19T01:42:29.909 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial keys... 2024-03-19T01:42:30.832 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQCG7fhlfgsLDxAAsWuT/QXiqdtv8Vbjvwyb+A== 2024-03-19T01:42:31.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQCH7fhlTqd6CBAAH3YWtZpQ5m9XesC4U6tZzw== 2024-03-19T01:42:32.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph-authtool: stdout AQCI7fhlRHZbARAAEkEWDFWj+80WRlhPeSm7sg== 2024-03-19T01:42:32.552 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial monmap... 2024-03-19T01:42:38.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-19T01:42:38.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-19T01:42:38.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:42:38.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-19T01:42:38.950 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool for a [v2:172.21.15.92:3300,v1:172.21.15.92:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-19T01:42:38.950 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = quincy 2024-03-19T01:42:38.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: set fsid to d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:42:38.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-19T01:42:38.950 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:42:38.950 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mon... 2024-03-19T01:42:43.537 INFO:teuthology.orchestra.run.smithi092.stdout:create mon.a on 2024-03-19T01:42:43.739 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-19T01:42:43.882 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-19T01:42:44.067 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d1e857f4-e591-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-d1e857f4-e591-11ee-95c9-87774f69a715.target. 2024-03-19T01:42:44.067 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d1e857f4-e591-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-d1e857f4-e591-11ee-95c9-87774f69a715.target. 2024-03-19T01:42:44.302 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.a 2024-03-19T01:42:44.302 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-19T01:42:44.460 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d1e857f4-e591-11ee-95c9-87774f69a715.target.wants/ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-d1e857f4-e591-11ee-95c9-87774f69a715@.service. 2024-03-19T01:42:44.977 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T01:42:44.977 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-03-19T01:42:44.983 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T01:42:44.984 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-03-19T01:42:44.984 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-03-19T01:42:44.984 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-03-19T01:42:44.984 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon to start... 2024-03-19T01:42:44.984 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mon... 2024-03-19T01:42:46.029 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout cluster: 2024-03-19T01:42:46.030 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout id: d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:42:46.030 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-19T01:42:46.030 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:42:46.030 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout services: 2024-03-19T01:42:46.031 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.486794s) 2024-03-19T01:42:46.031 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-19T01:42:46.031 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-19T01:42:46.031 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:42:46.031 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout data: 2024-03-19T01:42:46.031 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-19T01:42:46.032 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-19T01:42:46.032 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-19T01:42:46.032 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout pgs: 2024-03-19T01:42:46.032 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:42:46.032 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.372+0000 7ff7367e1700 1 Processor -- start 2024-03-19T01:42:46.032 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.372+0000 7ff7367e1700 1 -- start start 2024-03-19T01:42:46.033 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.372+0000 7ff7367e1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73007c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:46.033 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.372+0000 7ff7367e1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff73007c830 con 0x7ff73007de00 2024-03-19T01:42:46.033 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.372+0000 7ff72ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73007c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:46.033 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.372+0000 7ff72ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73007c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46458/0 (socket says 172.21.15.92:46458) 2024-03-19T01:42:46.033 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.372+0000 7ff72ffff700 1 -- 172.21.15.92:0/1723121680 learned_addr learned my addr 172.21.15.92:0/1723121680 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:46.033 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.373+0000 7ff72ffff700 1 -- 172.21.15.92:0/1723121680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73007c9b0 con 0x7ff73007de00 2024-03-19T01:42:46.034 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.373+0000 7ff72ffff700 1 --2- 172.21.15.92:0/1723121680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73007c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7200098a0 tx=0x7ff720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd1f20143881ebe server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:46.034 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.373+0000 7ff72effd700 1 -- 172.21.15.92:0/1723121680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff720004160 con 0x7ff73007de00 2024-03-19T01:42:46.034 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.373+0000 7ff72effd700 1 -- 172.21.15.92:0/1723121680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff720004300 con 0x7ff73007de00 2024-03-19T01:42:46.034 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.373+0000 7ff72effd700 1 -- 172.21.15.92:0/1723121680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7200036a0 con 0x7ff73007de00 2024-03-19T01:42:46.034 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.373+0000 7ff7367e1700 1 -- 172.21.15.92:0/1723121680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 msgr2=0x7ff73007c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:46.034 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.373+0000 7ff7367e1700 1 --2- 172.21.15.92:0/1723121680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73007c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7200098a0 tx=0x7ff720009bb0 comp rx=0 tx=0).stop 2024-03-19T01:42:46.035 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.374+0000 7ff7367e1700 1 -- 172.21.15.92:0/1723121680 shutdown_connections 2024-03-19T01:42:46.035 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.374+0000 7ff7367e1700 1 --2- 172.21.15.92:0/1723121680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73007c260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:46.035 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.374+0000 7ff7367e1700 1 -- 172.21.15.92:0/1723121680 >> 172.21.15.92:0/1723121680 conn(0x7ff730103200 msgr2=0x7ff730105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:46.035 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.374+0000 7ff7367e1700 1 -- 172.21.15.92:0/1723121680 shutdown_connections 2024-03-19T01:42:46.035 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.374+0000 7ff7367e1700 1 -- 172.21.15.92:0/1723121680 wait complete. 2024-03-19T01:42:46.035 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.374+0000 7ff7367e1700 1 Processor -- start 2024-03-19T01:42:46.036 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.374+0000 7ff7367e1700 1 -- start start 2024-03-19T01:42:46.036 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.374+0000 7ff7367e1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:46.036 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff7367e1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff73007cee0 con 0x7ff73007de00 2024-03-19T01:42:46.036 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff72ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:46.036 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff72ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46474/0 (socket says 172.21.15.92:46474) 2024-03-19T01:42:46.036 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff72ffff700 1 -- 172.21.15.92:0/2549293568 learned_addr learned my addr 172.21.15.92:0/2549293568 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:46.037 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff72ffff700 1 -- 172.21.15.92:0/2549293568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73019dac0 con 0x7ff73007de00 2024-03-19T01:42:46.037 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff72ffff700 1 --2- 172.21.15.92:0/2549293568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73019d580 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff720004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:46.037 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff72d7fa700 1 -- 172.21.15.92:0/2549293568 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff720003db0 con 0x7ff73007de00 2024-03-19T01:42:46.037 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff72d7fa700 1 -- 172.21.15.92:0/2549293568 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff720011630 con 0x7ff73007de00 2024-03-19T01:42:46.037 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff7367e1700 1 -- 172.21.15.92:0/2549293568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73019dd50 con 0x7ff73007de00 2024-03-19T01:42:46.037 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff72d7fa700 1 -- 172.21.15.92:0/2549293568 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff720011950 con 0x7ff73007de00 2024-03-19T01:42:46.037 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff7367e1700 1 -- 172.21.15.92:0/2549293568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7301a09f0 con 0x7ff73007de00 2024-03-19T01:42:46.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.375+0000 7ff72d7fa700 1 -- 172.21.15.92:0/2549293568 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff72001a460 con 0x7ff73007de00 2024-03-19T01:42:46.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.376+0000 7ff72d7fa700 1 -- 172.21.15.92:0/2549293568 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff72001b780 con 0x7ff73007de00 2024-03-19T01:42:46.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.376+0000 7ff7367e1700 1 -- 172.21.15.92:0/2549293568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff730063940 con 0x7ff73007de00 2024-03-19T01:42:46.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.379+0000 7ff72d7fa700 1 -- 172.21.15.92:0/2549293568 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff720031020 con 0x7ff73007de00 2024-03-19T01:42:46.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.447+0000 7ff7367e1700 1 -- 172.21.15.92:0/2549293568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff73002dd00 con 0x7ff73007de00 2024-03-19T01:42:46.038 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.448+0000 7ff72d7fa700 1 -- 172.21.15.92:0/2549293568 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7ff72001b980 con 0x7ff73007de00 2024-03-19T01:42:46.039 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.448+0000 7ff7367e1700 1 -- 172.21.15.92:0/2549293568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 msgr2=0x7ff73019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:46.039 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.448+0000 7ff7367e1700 1 --2- 172.21.15.92:0/2549293568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73019d580 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff720004750 comp rx=0 tx=0).stop 2024-03-19T01:42:46.039 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.449+0000 7ff7367e1700 1 -- 172.21.15.92:0/2549293568 shutdown_connections 2024-03-19T01:42:46.039 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.449+0000 7ff7367e1700 1 --2- 172.21.15.92:0/2549293568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73007de00 0x7ff73019d580 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:46.039 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.449+0000 7ff7367e1700 1 -- 172.21.15.92:0/2549293568 >> 172.21.15.92:0/2549293568 conn(0x7ff730103200 msgr2=0x7ff730105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:46.039 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.449+0000 7ff7367e1700 1 -- 172.21.15.92:0/2549293568 shutdown_connections 2024-03-19T01:42:46.039 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:45.449+0000 7ff7367e1700 1 -- 172.21.15.92:0/2549293568 wait complete. 2024-03-19T01:42:46.040 INFO:teuthology.orchestra.run.smithi092.stdout:mon is available 2024-03-19T01:42:46.040 INFO:teuthology.orchestra.run.smithi092.stdout:Assimilating anything we can from ceph.conf... 2024-03-19T01:42:47.110 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:46 smithi092 ceph-mon[62208]: from='client.? 172.21.15.92:0/2199963253' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-19T01:42:47.118 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:42:47.118 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-03-19T01:42:47.118 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:42:47.118 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.92:3300,v1:172.21.15.92:6789] 2024-03-19T01:42:47.118 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-19T01:42:47.118 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-19T01:42:47.118 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.455+0000 7fdfcdf6c700 1 Processor -- start 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.455+0000 7fdfcdf6c700 1 -- start start 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfcdf6c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc8106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:47.119 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfcdf6c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8107590 con 0x7fdfc8106ba0 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfc77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc8106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfc77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc8106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46490/0 (socket says 172.21.15.92:46490) 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfc77fe700 1 -- 172.21.15.92:0/4193378096 learned_addr learned my addr 172.21.15.92:0/4193378096 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfc77fe700 1 -- 172.21.15.92:0/4193378096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc8107da0 con 0x7fdfc8106ba0 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfc77fe700 1 --2- 172.21.15.92:0/4193378096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc8106fc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80098a0 tx=0x7fdfb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e21f26ab5016d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfc6ffd700 1 -- 172.21.15.92:0/4193378096 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb8004160 con 0x7fdfc8106ba0 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfc6ffd700 1 -- 172.21.15.92:0/4193378096 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb8004300 con 0x7fdfc8106ba0 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.456+0000 7fdfc6ffd700 1 -- 172.21.15.92:0/4193378096 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb80036a0 con 0x7fdfc8106ba0 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.457+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/4193378096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 msgr2=0x7fdfc8106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.457+0000 7fdfcdf6c700 1 --2- 172.21.15.92:0/4193378096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc8106fc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80098a0 tx=0x7fdfb8009bb0 comp rx=0 tx=0).stop 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.457+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/4193378096 shutdown_connections 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.457+0000 7fdfcdf6c700 1 --2- 172.21.15.92:0/4193378096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc8106fc0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:47.120 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.457+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/4193378096 >> 172.21.15.92:0/4193378096 conn(0x7fdfc8102140 msgr2=0x7fdfc8104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.457+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/4193378096 shutdown_connections 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.457+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/4193378096 wait complete. 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.457+0000 7fdfcdf6c700 1 Processor -- start 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.457+0000 7fdfcdf6c700 1 -- start start 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfcdf6c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc818f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfcdf6c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc81082d0 con 0x7fdfc8106ba0 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfc77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc818f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfc77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc818f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46506/0 (socket says 172.21.15.92:46506) 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfc77fe700 1 -- 172.21.15.92:0/2199963253 learned_addr learned my addr 172.21.15.92:0/2199963253 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfc77fe700 1 -- 172.21.15.92:0/2199963253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc818fc50 con 0x7fdfc8106ba0 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfc77fe700 1 --2- 172.21.15.92:0/2199963253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc818f710 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8000c00 tx=0x7fdfb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfc57fa700 1 -- 172.21.15.92:0/2199963253 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb8003db0 con 0x7fdfc8106ba0 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfc57fa700 1 -- 172.21.15.92:0/2199963253 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb8011630 con 0x7fdfc8106ba0 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/2199963253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc818de90 con 0x7fdfc8106ba0 2024-03-19T01:42:47.121 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.458+0000 7fdfc57fa700 1 -- 172.21.15.92:0/2199963253 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfb8011950 con 0x7fdfc8106ba0 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.459+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/2199963253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc818e370 con 0x7fdfc8106ba0 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.459+0000 7fdfc57fa700 1 -- 172.21.15.92:0/2199963253 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdfb8011af0 con 0x7fdfc8106ba0 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.459+0000 7fdfc57fa700 1 -- 172.21.15.92:0/2199963253 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdfb801c450 con 0x7fdfc8106ba0 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.459+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/2199963253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc8063940 con 0x7fdfc8106ba0 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.462+0000 7fdfc57fa700 1 -- 172.21.15.92:0/2199963253 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdfb8032020 con 0x7fdfc8106ba0 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.529+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/2199963253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fdfc8194b30 con 0x7fdfc8106ba0 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.532+0000 7fdfc57fa700 1 -- 172.21.15.92:0/2199963253 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fdfb8004260 con 0x7fdfc8106ba0 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.533+0000 7fdfc57fa700 1 -- 172.21.15.92:0/2199963253 <== mon.0 v2:172.21.15.92:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdfb801cb60 con 0x7fdfc8106ba0 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.534+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/2199963253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 msgr2=0x7fdfc818f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.534+0000 7fdfcdf6c700 1 --2- 172.21.15.92:0/2199963253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc818f710 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8000c00 tx=0x7fdfb8004750 comp rx=0 tx=0).stop 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.534+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/2199963253 shutdown_connections 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.534+0000 7fdfcdf6c700 1 --2- 172.21.15.92:0/2199963253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfc8106ba0 0x7fdfc818f710 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.534+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/2199963253 >> 172.21.15.92:0/2199963253 conn(0x7fdfc8102140 msgr2=0x7fdfc8191fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.534+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/2199963253 shutdown_connections 2024-03-19T01:42:47.122 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:46.534+0000 7fdfcdf6c700 1 -- 172.21.15.92:0/2199963253 wait complete. 2024-03-19T01:42:47.123 INFO:teuthology.orchestra.run.smithi092.stdout:Generating new minimal ceph.conf... 2024-03-19T01:42:48.184 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:47 smithi092 ceph-mon[62208]: from='client.? 172.21.15.92:0/3666470732' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:42:48.190 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.543+0000 7f7c763f4700 1 Processor -- start 2024-03-19T01:42:48.190 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.543+0000 7f7c763f4700 1 -- start start 2024-03-19T01:42:48.190 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c763f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c70107470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:48.190 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c763f4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c70107a40 con 0x7f7c70107050 2024-03-19T01:42:48.190 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c70107470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c70107470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46520/0 (socket says 172.21.15.92:46520) 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c6ffff700 1 -- 172.21.15.92:0/853031811 learned_addr learned my addr 172.21.15.92:0/853031811 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c6ffff700 1 -- 172.21.15.92:0/853031811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c70107d70 con 0x7f7c70107050 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c6ffff700 1 --2- 172.21.15.92:0/853031811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c70107470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7c580098a0 tx=0x7f7c58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350eedc479fd0206 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c6effd700 1 -- 172.21.15.92:0/853031811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c58004160 con 0x7f7c70107050 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c6effd700 1 -- 172.21.15.92:0/853031811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c58004300 con 0x7f7c70107050 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.544+0000 7f7c6effd700 1 -- 172.21.15.92:0/853031811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c58011b50 con 0x7f7c70107050 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.545+0000 7f7c763f4700 1 -- 172.21.15.92:0/853031811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 msgr2=0x7f7c70107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.545+0000 7f7c763f4700 1 --2- 172.21.15.92:0/853031811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c70107470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7c580098a0 tx=0x7f7c58009bb0 comp rx=0 tx=0).stop 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.545+0000 7f7c763f4700 1 -- 172.21.15.92:0/853031811 shutdown_connections 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.545+0000 7f7c763f4700 1 --2- 172.21.15.92:0/853031811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c70107470 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.545+0000 7f7c763f4700 1 -- 172.21.15.92:0/853031811 >> 172.21.15.92:0/853031811 conn(0x7f7c70101890 msgr2=0x7f7c70103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.545+0000 7f7c763f4700 1 -- 172.21.15.92:0/853031811 shutdown_connections 2024-03-19T01:42:48.191 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.545+0000 7f7c763f4700 1 -- 172.21.15.92:0/853031811 wait complete. 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.546+0000 7f7c763f4700 1 Processor -- start 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.546+0000 7f7c763f4700 1 -- start start 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.546+0000 7f7c763f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c7019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.546+0000 7f7c763f4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7010b120 con 0x7f7c70107050 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.546+0000 7f7c6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c7019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.546+0000 7f7c6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c7019bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46522/0 (socket says 172.21.15.92:46522) 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.546+0000 7f7c6ffff700 1 -- 172.21.15.92:0/3666470732 learned_addr learned my addr 172.21.15.92:0/3666470732 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.546+0000 7f7c6ffff700 1 -- 172.21.15.92:0/3666470732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7019c160 con 0x7f7c70107050 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.546+0000 7f7c6ffff700 1 --2- 172.21.15.92:0/3666470732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c7019bc20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009fb0 tx=0x7f7c58011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.547+0000 7f7c6d7fa700 1 -- 172.21.15.92:0/3666470732 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c580038b0 con 0x7f7c70107050 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.547+0000 7f7c763f4700 1 -- 172.21.15.92:0/3666470732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7019c3f0 con 0x7f7c70107050 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.547+0000 7f7c6d7fa700 1 -- 172.21.15.92:0/3666470732 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c58011e50 con 0x7f7c70107050 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.547+0000 7f7c6d7fa700 1 -- 172.21.15.92:0/3666470732 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c58019420 con 0x7f7c70107050 2024-03-19T01:42:48.192 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.547+0000 7f7c763f4700 1 -- 172.21.15.92:0/3666470732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c7019f0e0 con 0x7f7c70107050 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.547+0000 7f7c6d7fa700 1 -- 172.21.15.92:0/3666470732 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7c58019b30 con 0x7f7c70107050 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.547+0000 7f7c6d7fa700 1 -- 172.21.15.92:0/3666470732 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7c58024730 con 0x7f7c70107050 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.548+0000 7f7c763f4700 1 -- 172.21.15.92:0/3666470732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c70063940 con 0x7f7c70107050 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.550+0000 7f7c6d7fa700 1 -- 172.21.15.92:0/3666470732 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7c58003a50 con 0x7f7c70107050 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.617+0000 7f7c763f4700 1 -- 172.21.15.92:0/3666470732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7c700008d0 con 0x7f7c70107050 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.618+0000 7f7c6d7fa700 1 -- 172.21.15.92:0/3666470732 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f7c580249d0 con 0x7f7c70107050 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.618+0000 7f7c763f4700 1 -- 172.21.15.92:0/3666470732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 msgr2=0x7f7c7019bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.618+0000 7f7c763f4700 1 --2- 172.21.15.92:0/3666470732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c7019bc20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009fb0 tx=0x7f7c58011bf0 comp rx=0 tx=0).stop 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.619+0000 7f7c763f4700 1 -- 172.21.15.92:0/3666470732 shutdown_connections 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.619+0000 7f7c763f4700 1 --2- 172.21.15.92:0/3666470732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c70107050 0x7f7c7019bc20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.619+0000 7f7c763f4700 1 -- 172.21.15.92:0/3666470732 >> 172.21.15.92:0/3666470732 conn(0x7f7c70101890 msgr2=0x7f7c701027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.619+0000 7f7c763f4700 1 -- 172.21.15.92:0/3666470732 shutdown_connections 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:47.619+0000 7f7c763f4700 1 -- 172.21.15.92:0/3666470732 wait complete. 2024-03-19T01:42:48.193 INFO:teuthology.orchestra.run.smithi092.stdout:Restarting the monitor... 2024-03-19T01:42:48.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:48 smithi092 systemd[1]: Stopping Ceph mon.a for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:42:49.702 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:49 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a[62204]: 2024-03-19T01:42:49.317+0000 7ff36da2f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T01:42:49.702 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:49 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a[62204]: 2024-03-19T01:42:49.317+0000 7ff36da2f700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-19T01:42:49.702 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:49 smithi092 podman[62451]: 2024-03-19 01:42:49.461484438 +0000 UTC m=+1.111101865 container died 698aea375cf32d340d7235d1e7f85b9790bad7fd3dc39df22fb4b75f8197acdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-19T01:42:49.977 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:49 smithi092 podman[62451]: 2024-03-19 01:42:49.776547627 +0000 UTC m=+1.426165048 container cleanup 698aea375cf32d340d7235d1e7f85b9790bad7fd3dc39df22fb4b75f8197acdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-19T01:42:49.977 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:49 smithi092 bash[62451]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a 2024-03-19T01:42:50.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:49 smithi092 podman[62465]: 2024-03-19 01:42:49.977571366 +0000 UTC m=+0.510623960 container remove 698aea375cf32d340d7235d1e7f85b9790bad7fd3dc39df22fb4b75f8197acdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS) 2024-03-19T01:42:50.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-19T01:42:50.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 systemd[1]: Stopped Ceph mon.a for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:42:50.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 systemd[1]: Starting Ceph mon.a for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 podman[62555]: 2024-03-19 01:42:50.312808824 +0000 UTC m=+0.017915900 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 podman[62555]: 2024-03-19 01:42:50.420146774 +0000 UTC m=+0.125253904 container create 9b3aea2ff4cd4cee522905e2fa6c6b332d63335d29da091fe46ffaa83344b750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True) 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 podman[62555]: 2024-03-19 01:42:50.589210443 +0000 UTC m=+0.294317511 container init 9b3aea2ff4cd4cee522905e2fa6c6b332d63335d29da091fe46ffaa83344b750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 podman[62555]: 2024-03-19 01:42:50.591936182 +0000 UTC m=+0.297043244 container start 9b3aea2ff4cd4cee522905e2fa6c6b332d63335d29da091fe46ffaa83344b750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: set uid:gid to 167:167 (ceph:ceph) 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: pidfile_write: ignore empty --pid-file 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: load: jerasure load: lrc 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: RocksDB version: 7.9.2 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Git sha 0 2024-03-19T01:42:50.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-19T01:42:50.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: DB SUMMARY 2024-03-19T01:42:50.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: DB Session ID: 11LHJ13P0QFHMR074OG0 2024-03-19T01:42:50.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: CURRENT file: CURRENT 2024-03-19T01:42:50.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: IDENTITY file: IDENTITY 2024-03-19T01:42:50.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-19T01:42:50.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-19T01:42:50.665 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104338 ; 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.error_if_exists: 0 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.create_if_missing: 0 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.paranoid_checks: 1 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.env: 0x564e940f7c60 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.fs: PosixFileSystem 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.info_log: 0x564e9505dae0 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_file_opening_threads: 16 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.statistics: (nil) 2024-03-19T01:42:50.666 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.use_fsync: 0 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_log_file_size: 0 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.keep_log_file_num: 1000 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.recycle_log_file_num: 0 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.allow_fallocate: 1 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.allow_mmap_reads: 0 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.allow_mmap_writes: 0 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.use_direct_reads: 0 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.create_missing_column_families: 0 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.db_log_dir: 2024-03-19T01:42:50.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.wal_dir: 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.advise_random_on_open: 1 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.db_write_buffer_size: 0 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.write_buffer_manager: 0x564e950301e0 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-19T01:42:50.668 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.rate_limiter: (nil) 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.wal_recovery_mode: 2 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.enable_thread_tracking: 0 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.enable_pipelined_write: 0 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.unordered_write: 0 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-19T01:42:50.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.row_cache: None 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.wal_filter: None 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.allow_ingest_behind: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.two_write_queues: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.manual_wal_flush: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.wal_compression: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.atomic_flush: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.log_readahead_size: 0 2024-03-19T01:42:50.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.best_efforts_recovery: 0 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.allow_data_in_errors: 0 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.db_host_id: __hostname__ 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_background_jobs: 2 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_background_compactions: -1 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_subcompactions: 1 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_total_wal_size: 0 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-19T01:42:50.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_open_files: -1 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bytes_per_sync: 0 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_readahead_size: 0 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_background_flushes: -1 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Compression algorithms supported: 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: kZSTD supported: 0 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: kXpressCompression supported: 0 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: kLZ4HCCompression supported: 1 2024-03-19T01:42:50.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: kZlibCompression supported: 1 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: kSnappyCompression supported: 1 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: kLZ4Compression supported: 1 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: kBZip2Compression supported: 0 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.merge_operator: 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_filter: None 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_filter_factory: None 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.sst_partitioner_factory: None 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564e9505dbe0) 2024-03-19T01:42:50.673 INFO:journalctl@ceph.mon.a.smithi092.stdout: cache_index_and_filter_blocks: 1 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: pin_top_level_index_and_filter: 1 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_type: 0 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: data_block_index_type: 0 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_shortening: 1 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: checksum: 4 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: no_block_cache: 0 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache: 0x564e95063090 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_name: BinnedLRUCache 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_options: 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: capacity : 536870912 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: num_shard_bits : 4 2024-03-19T01:42:50.674 INFO:journalctl@ceph.mon.a.smithi092.stdout: strict_capacity_limit : 0 2024-03-19T01:42:50.675 INFO:journalctl@ceph.mon.a.smithi092.stdout: high_pri_pool_ratio: 0.000 2024-03-19T01:42:50.675 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_cache_compressed: (nil) 2024-03-19T01:42:50.675 INFO:journalctl@ceph.mon.a.smithi092.stdout: persistent_cache: (nil) 2024-03-19T01:42:50.675 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_size: 4096 2024-03-19T01:42:50.675 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_size_deviation: 10 2024-03-19T01:42:50.675 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_restart_interval: 16 2024-03-19T01:42:50.675 INFO:journalctl@ceph.mon.a.smithi092.stdout: index_block_restart_interval: 1 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: metadata_block_size: 4096 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: partition_filters: 0 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: use_delta_encoding: 1 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: filter_policy: bloomfilter 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: whole_key_filtering: 1 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: verify_compression: 0 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: read_amp_bytes_per_bit: 0 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: format_version: 5 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: enable_index_compression: 1 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: block_align: 0 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: max_auto_readahead_size: 262144 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: prepopulate_block_cache: 0 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: initial_auto_readahead_size: 8192 2024-03-19T01:42:50.676 INFO:journalctl@ceph.mon.a.smithi092.stdout: num_file_reads_for_auto_readahead: 2 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.write_buffer_size: 33554432 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_write_buffer_number: 2 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression: NoCompression 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression: Disabled 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.prefix_extractor: nullptr 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.num_levels: 7 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-19T01:42:50.677 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression_opts.level: 32767 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression_opts.strategy: 0 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression_opts.enabled: false 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-19T01:42:50.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.target_file_size_base: 67108864 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-19T01:42:50.679 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.arena_block_size: 1048576 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.disable_auto_compactions: 0 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-19T01:42:50.680 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.table_properties_collectors: 2024-03-19T01:42:50.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.inplace_update_support: 0 2024-03-19T01:42:50.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-19T01:42:50.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-19T01:42:50.681 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-19T01:42:50.681 INFO:teuthology.orchestra.run.smithi092.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.bloom_locality: 0 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.max_successive_merges: 0 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.paranoid_file_checks: 0 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.force_consistency_checks: 1 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.report_bg_io_stats: 0 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.ttl: 2592000 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.enable_blob_files: false 2024-03-19T01:42:50.682 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.min_blob_size: 0 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.blob_file_size: 268435456 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.blob_file_starting_level: 0 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b0425f5c-4912-4b90-b689-b0e28a6532fb 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710812570621351, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710812570622767, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99505, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97477, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13975, "raw_average_key_size": 49, "raw_value_size": 89848, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710812570, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b0425f5c-4912-4b90-b689-b0e28a6532fb", "db_session_id": "11LHJ13P0QFHMR074OG0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-19T01:42:50.683 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710812570622876, "job": 1, "event": "recovery_finished"} 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564e95156000 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: DB pointer 0x564e95140000 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** DB Stats ** 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-19T01:42:50.684 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: L0 2/0 99.01 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 114.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: Sum 2/0 99.01 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 114.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 114.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-03-19T01:42:50.685 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** Compaction Stats [default] ** 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 114.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: Cumulative compaction: 0.00 GB write, 22.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T01:42:50.686 INFO:journalctl@ceph.mon.a.smithi092.stdout: Interval compaction: 0.00 GB write, 22.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: Block cache BinnedLRUCache@0x564e95063090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: starting mon.a rank 0 at public addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] at bind addrs [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mon.a@-1(???) e1 preinit fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mon.a@-1(???).mds e1 new map 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mon.a@-1(???).mds e1 print_map 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: e1 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: legacy client fscid: -1 2024-03-19T01:42:50.687 INFO:journalctl@ceph.mon.a.smithi092.stdout: 2024-03-19T01:42:50.688 INFO:journalctl@ceph.mon.a.smithi092.stdout: No filesystems configured 2024-03-19T01:42:50.688 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-19T01:42:50.688 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:42:50.688 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:42:50.688 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:42:50.688 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-19T01:42:50.688 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 bash[62555]: 9b3aea2ff4cd4cee522905e2fa6c6b332d63335d29da091fe46ffaa83344b750 2024-03-19T01:42:50.688 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 systemd[1]: Started Ceph mon.a for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:42:50.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-19T01:42:50.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: monmap epoch 1 2024-03-19T01:42:50.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:42:50.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: last_changed 2024-03-19T01:42:32.859371+0000 2024-03-19T01:42:50.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: created 2024-03-19T01:42:32.859371+0000 2024-03-19T01:42:50.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: min_mon_release 19 (squid) 2024-03-19T01:42:50.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: election_strategy: 1 2024-03-19T01:42:50.933 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-03-19T01:42:50.934 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: fsmap 2024-03-19T01:42:50.934 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: osdmap e1: 0 total, 0 up, 0 in 2024-03-19T01:42:50.934 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:50 smithi092 ceph-mon[62574]: mgrmap e1: no daemons active 2024-03-19T01:42:51.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.033+0000 7f5162824700 1 Processor -- start 2024-03-19T01:42:51.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.033+0000 7f5162824700 1 -- start start 2024-03-19T01:42:51.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.033+0000 7f5162824700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c106360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:51.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.033+0000 7f5162824700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f515c106930 con 0x7f515c105f40 2024-03-19T01:42:51.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.033+0000 7f515bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c106360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:51.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.033+0000 7f515bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c106360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46534/0 (socket says 172.21.15.92:46534) 2024-03-19T01:42:51.618 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.033+0000 7f515bfff700 1 -- 172.21.15.92:0/4167641593 learned_addr learned my addr 172.21.15.92:0/4167641593 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f515bfff700 1 -- 172.21.15.92:0/4167641593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c108ed0 con 0x7f515c105f40 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f515bfff700 1 --2- 172.21.15.92:0/4167641593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c106360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f514c0098a0 tx=0x7f514c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf0b6f2d536af63 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f515affd700 1 -- 172.21.15.92:0/4167641593 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f514c004160 con 0x7f515c105f40 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f515affd700 1 -- 172.21.15.92:0/4167641593 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f514c004300 con 0x7f515c105f40 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f515affd700 1 -- 172.21.15.92:0/4167641593 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f514c011b50 con 0x7f515c105f40 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f5162824700 1 -- 172.21.15.92:0/4167641593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 msgr2=0x7f515c106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f5162824700 1 --2- 172.21.15.92:0/4167641593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c106360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f514c0098a0 tx=0x7f514c009bb0 comp rx=0 tx=0).stop 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f5162824700 1 -- 172.21.15.92:0/4167641593 shutdown_connections 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f5162824700 1 --2- 172.21.15.92:0/4167641593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c106360 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.034+0000 7f5162824700 1 -- 172.21.15.92:0/4167641593 >> 172.21.15.92:0/4167641593 conn(0x7f515c101890 msgr2=0x7f515c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.035+0000 7f5162824700 1 -- 172.21.15.92:0/4167641593 shutdown_connections 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.035+0000 7f5162824700 1 -- 172.21.15.92:0/4167641593 wait complete. 2024-03-19T01:42:51.619 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.035+0000 7f5162824700 1 Processor -- start 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.035+0000 7f5162824700 1 -- start start 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.035+0000 7f5162824700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.035+0000 7f5162824700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10c280 con 0x7f515c105f40 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f515bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f515bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c19bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46542/0 (socket says 172.21.15.92:46542) 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f515bfff700 1 -- 172.21.15.92:0/1094144497 learned_addr learned my addr 172.21.15.92:0/1094144497 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f515bfff700 1 -- 172.21.15.92:0/1094144497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c19c1b0 con 0x7f515c105f40 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f515bfff700 1 --2- 172.21.15.92:0/1094144497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c19bc70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f514c011850 tx=0x7f514c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f51597fa700 1 -- 172.21.15.92:0/1094144497 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f514c003af0 con 0x7f515c105f40 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f51597fa700 1 -- 172.21.15.92:0/1094144497 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f514c020070 con 0x7f515c105f40 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f5162824700 1 -- 172.21.15.92:0/1094144497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c19c440 con 0x7f515c105f40 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f51597fa700 1 -- 172.21.15.92:0/1094144497 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f514c0195a0 con 0x7f515c105f40 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f5162824700 1 -- 172.21.15.92:0/1094144497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c19f130 con 0x7f515c105f40 2024-03-19T01:42:51.620 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f51597fa700 1 -- 172.21.15.92:0/1094144497 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f514c019c30 con 0x7f515c105f40 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.036+0000 7f51597fa700 1 -- 172.21.15.92:0/1094144497 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f514c011e00 con 0x7f515c105f40 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.037+0000 7f5162824700 1 -- 172.21.15.92:0/1094144497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c063940 con 0x7f515c105f40 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.039+0000 7f51597fa700 1 -- 172.21.15.92:0/1094144497 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f514c003c90 con 0x7f515c105f40 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.106+0000 7f5162824700 1 -- 172.21.15.92:0/1094144497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f515c19f300 con 0x7f515c105f40 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.108+0000 7f51597fa700 1 -- 172.21.15.92:0/1094144497 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f514c01a430 con 0x7f515c105f40 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.108+0000 7f5162824700 1 -- 172.21.15.92:0/1094144497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 msgr2=0x7f515c19bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.108+0000 7f5162824700 1 --2- 172.21.15.92:0/1094144497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c19bc70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f514c011850 tx=0x7f514c011c10 comp rx=0 tx=0).stop 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.108+0000 7f5162824700 1 -- 172.21.15.92:0/1094144497 shutdown_connections 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.108+0000 7f5162824700 1 --2- 172.21.15.92:0/1094144497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f515c105f40 0x7f515c19bc70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.108+0000 7f5162824700 1 -- 172.21.15.92:0/1094144497 >> 172.21.15.92:0/1094144497 conn(0x7f515c101890 msgr2=0x7f515c102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.108+0000 7f5162824700 1 -- 172.21.15.92:0/1094144497 shutdown_connections 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:51.109+0000 7f5162824700 1 -- 172.21.15.92:0/1094144497 wait complete. 2024-03-19T01:42:51.621 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-19T01:42:51.627 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-19T01:42:51.627 INFO:teuthology.orchestra.run.smithi092.stdout:Creating mgr... 2024-03-19T01:42:51.628 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-19T01:42:51.628 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-19T01:42:51.857 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.x 2024-03-19T01:42:51.857 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-19T01:42:52.001 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d1e857f4-e591-11ee-95c9-87774f69a715.target.wants/ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-d1e857f4-e591-11ee-95c9-87774f69a715@.service. 2024-03-19T01:42:52.020 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:52 smithi092 systemd[1]: Starting Ceph mgr.x for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:42:52.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:52 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1094144497' entity='client.admin' 2024-03-19T01:42:52.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:52 smithi092 podman[62790]: 2024-03-19 01:42:52.13995247 +0000 UTC m=+0.018254969 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 2024-03-19T01:42:52.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:52 smithi092 podman[62790]: 2024-03-19 01:42:52.264396293 +0000 UTC m=+0.142698837 container create 56cbeab0cd4c89214867b5094094ea0c63c8da43ccad7ea8a4a19ec66ca9b5b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-19T01:42:52.521 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T01:42:52.521 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-03-19T01:42:52.528 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T01:42:52.528 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-03-19T01:42:52.528 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-03-19T01:42:52.528 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to enable service . firewalld.service is not available 2024-03-19T01:42:52.536 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T01:42:52.536 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-03-19T01:42:52.543 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T01:42:52.543 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-03-19T01:42:52.543 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-03-19T01:42:52.543 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-19T01:42:52.543 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr to start... 2024-03-19T01:42:52.543 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr... 2024-03-19T01:42:52.668 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:52 smithi092 podman[62790]: 2024-03-19 01:42:52.449925601 +0000 UTC m=+0.328228099 container init 56cbeab0cd4c89214867b5094094ea0c63c8da43ccad7ea8a4a19ec66ca9b5b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-19T01:42:52.668 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:52 smithi092 podman[62790]: 2024-03-19 01:42:52.452287435 +0000 UTC m=+0.330589934 container start 56cbeab0cd4c89214867b5094094ea0c63c8da43ccad7ea8a4a19ec66ca9b5b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:42:52.668 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:52 smithi092 bash[62790]: 56cbeab0cd4c89214867b5094094ea0c63c8da43ccad7ea8a4a19ec66ca9b5b3 2024-03-19T01:42:52.668 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:52 smithi092 systemd[1]: Started Ceph mgr.x for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:42:53.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:53 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3182500662' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T01:42:53.311 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:53.020+0000 7f5f26405200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:42:53.311 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:53.119+0000 7f5f26405200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:42:53.659 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:42:53.661 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-03-19T01:42:53.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:42:53.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T01:42:53.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T01:42:53.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T01:42:53.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T01:42:53.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:53.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T01:42:53.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T01:42:53.662 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T01:42:53.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T01:42:53.671 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T01:42:53.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T01:42:44.963671+0000", 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.952+0000 7f0b48fdf700 1 Processor -- start 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.952+0000 7f0b48fdf700 1 -- start start 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.952+0000 7f0b48fdf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b44106370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.952+0000 7f0b48fdf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44106940 con 0x7f0b44105f50 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.952+0000 7f0b4259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b44106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.952+0000 7f0b4259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b44106370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46570/0 (socket says 172.21.15.92:46570) 2024-03-19T01:42:53.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.952+0000 7f0b4259c700 1 -- 172.21.15.92:0/2174594499 learned_addr learned my addr 172.21.15.92:0/2174594499 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.953+0000 7f0b4259c700 1 -- 172.21.15.92:0/2174594499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b44108ee0 con 0x7f0b44105f50 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.953+0000 7f0b4259c700 1 --2- 172.21.15.92:0/2174594499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b44106370 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0b340098a0 tx=0x7f0b34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ee663ca13cd5dc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.953+0000 7f0b4159a700 1 -- 172.21.15.92:0/2174594499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34004160 con 0x7f0b44105f50 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.953+0000 7f0b4159a700 1 -- 172.21.15.92:0/2174594499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b34004300 con 0x7f0b44105f50 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.953+0000 7f0b4159a700 1 -- 172.21.15.92:0/2174594499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34011b50 con 0x7f0b44105f50 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.954+0000 7f0b48fdf700 1 -- 172.21.15.92:0/2174594499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 msgr2=0x7f0b44106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.954+0000 7f0b48fdf700 1 --2- 172.21.15.92:0/2174594499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b44106370 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0b340098a0 tx=0x7f0b34009bb0 comp rx=0 tx=0).stop 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.954+0000 7f0b48fdf700 1 -- 172.21.15.92:0/2174594499 shutdown_connections 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.954+0000 7f0b48fdf700 1 --2- 172.21.15.92:0/2174594499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b44106370 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.954+0000 7f0b48fdf700 1 -- 172.21.15.92:0/2174594499 >> 172.21.15.92:0/2174594499 conn(0x7f0b44101880 msgr2=0x7f0b44103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.954+0000 7f0b48fdf700 1 -- 172.21.15.92:0/2174594499 shutdown_connections 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.954+0000 7f0b48fdf700 1 -- 172.21.15.92:0/2174594499 wait complete. 2024-03-19T01:42:53.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.955+0000 7f0b48fdf700 1 Processor -- start 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.955+0000 7f0b48fdf700 1 -- start start 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.955+0000 7f0b48fdf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b4419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.955+0000 7f0b48fdf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4410c290 con 0x7f0b44105f50 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.955+0000 7f0b4259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b4419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.955+0000 7f0b4259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b4419bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46582/0 (socket says 172.21.15.92:46582) 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.955+0000 7f0b4259c700 1 -- 172.21.15.92:0/3182500662 learned_addr learned my addr 172.21.15.92:0/3182500662 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.955+0000 7f0b4259c700 1 -- 172.21.15.92:0/3182500662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4419c160 con 0x7f0b44105f50 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.955+0000 7f0b4259c700 1 --2- 172.21.15.92:0/3182500662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b4419bc20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0b34011850 tx=0x7f0b34011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.956+0000 7f0b337fe700 1 -- 172.21.15.92:0/3182500662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34003af0 con 0x7f0b44105f50 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.956+0000 7f0b337fe700 1 -- 172.21.15.92:0/3182500662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b34020070 con 0x7f0b44105f50 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.956+0000 7f0b337fe700 1 -- 172.21.15.92:0/3182500662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b340195a0 con 0x7f0b44105f50 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.956+0000 7f0b48fdf700 1 -- 172.21.15.92:0/3182500662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4419c3f0 con 0x7f0b44105f50 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.956+0000 7f0b48fdf700 1 -- 172.21.15.92:0/3182500662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b4419f0e0 con 0x7f0b44105f50 2024-03-19T01:42:53.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.956+0000 7f0b337fe700 1 -- 172.21.15.92:0/3182500662 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0b34019be0 con 0x7f0b44105f50 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.956+0000 7f0b337fe700 1 -- 172.21.15.92:0/3182500662 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0b34011e00 con 0x7f0b44105f50 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.957+0000 7f0b48fdf700 1 -- 172.21.15.92:0/3182500662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b44063940 con 0x7f0b44105f50 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:52.959+0000 7f0b337fe700 1 -- 172.21.15.92:0/3182500662 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0b34003c90 con 0x7f0b44105f50 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:53.045+0000 7f0b48fdf700 1 -- 172.21.15.92:0/3182500662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0b4419f2b0 con 0x7f0b44105f50 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:53.045+0000 7f0b337fe700 1 -- 172.21.15.92:0/3182500662 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0b34025460 con 0x7f0b44105f50 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:53.046+0000 7f0b48fdf700 1 -- 172.21.15.92:0/3182500662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 msgr2=0x7f0b4419bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:53.046+0000 7f0b48fdf700 1 --2- 172.21.15.92:0/3182500662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b4419bc20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0b34011850 tx=0x7f0b34011c10 comp rx=0 tx=0).stop 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:53.046+0000 7f0b48fdf700 1 -- 172.21.15.92:0/3182500662 shutdown_connections 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:53.046+0000 7f0b48fdf700 1 --2- 172.21.15.92:0/3182500662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44105f50 0x7f0b4419bc20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:53.046+0000 7f0b48fdf700 1 -- 172.21.15.92:0/3182500662 >> 172.21.15.92:0/3182500662 conn(0x7f0b44101880 msgr2=0x7f0b441023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:53.046+0000 7f0b48fdf700 1 -- 172.21.15.92:0/3182500662 shutdown_connections 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:53.046+0000 7f0b48fdf700 1 -- 172.21.15.92:0/3182500662 wait complete. 2024-03-19T01:42:53.677 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (1/15)... 2024-03-19T01:42:54.006 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:53.772+0000 7f5f26405200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:42:54.006 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:53.864+0000 7f5f26405200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:42:54.006 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:54.005+0000 7f5f26405200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:42:54.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:54.101+0000 7f5f26405200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:42:54.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:54.184+0000 7f5f26405200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:42:54.734 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:54.446+0000 7f5f26405200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:42:54.734 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:54.554+0000 7f5f26405200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:42:54.734 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:54.638+0000 7f5f26405200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:42:54.734 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:54.734+0000 7f5f26405200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:42:55.019 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:54.906+0000 7f5f26405200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:42:56.053 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:55 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:55.917+0000 7f5f26405200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:42:56.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:56 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3928377312' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T01:42:56.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:56 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:56.052+0000 7f5f26405200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:42:56.685 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:56 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:56.400+0000 7f5f26405200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:42:56.685 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:56 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:56.489+0000 7f5f26405200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:42:56.685 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:56 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:56.684+0000 7f5f26405200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T01:42:56.876 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:56.877 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T01:42:56.878 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T01:42:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T01:42:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T01:42:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T01:42:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T01:42:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T01:42:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T01:42:56.887 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T01:42:44.963671+0000", 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T01:42:56.888 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.134+0000 7f8c3c99b700 1 Processor -- start 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.134+0000 7f8c3c99b700 1 -- start start 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3c99b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3810ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3c99b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3807bdf0 con 0x7f8c38108750 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3810ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3810ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41472/0 (socket says 172.21.15.92:41472) 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3659c700 1 -- 172.21.15.92:0/3310637819 learned_addr learned my addr 172.21.15.92:0/3310637819 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3659c700 1 -- 172.21.15.92:0/3310637819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3810b080 con 0x7f8c38108750 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3659c700 1 --2- 172.21.15.92:0/3310637819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3810ab40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009ab0 tx=0x7f8c28009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72cdfe92990a334 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3559a700 1 -- 172.21.15.92:0/3310637819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c28004160 con 0x7f8c38108750 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3559a700 1 -- 172.21.15.92:0/3310637819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c2800bca0 con 0x7f8c38108750 2024-03-19T01:42:56.889 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.135+0000 7f8c3559a700 1 -- 172.21.15.92:0/3310637819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c28003d60 con 0x7f8c38108750 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.136+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3310637819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 msgr2=0x7f8c3810ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.136+0000 7f8c3c99b700 1 --2- 172.21.15.92:0/3310637819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3810ab40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009ab0 tx=0x7f8c28009dc0 comp rx=0 tx=0).stop 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.136+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3310637819 shutdown_connections 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.136+0000 7f8c3c99b700 1 --2- 172.21.15.92:0/3310637819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3810ab40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.136+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3310637819 >> 172.21.15.92:0/3310637819 conn(0x7f8c38102140 msgr2=0x7f8c38104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.136+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3310637819 shutdown_connections 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.136+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3310637819 wait complete. 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.137+0000 7f8c3c99b700 1 Processor -- start 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.137+0000 7f8c3c99b700 1 -- start start 2024-03-19T01:42:56.890 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.137+0000 7f8c3c99b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.137+0000 7f8c3c99b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3810c030 con 0x7f8c38108750 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.137+0000 7f8c3659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.137+0000 7f8c3659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41478/0 (socket says 172.21.15.92:41478) 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.137+0000 7f8c3659c700 1 -- 172.21.15.92:0/3928377312 learned_addr learned my addr 172.21.15.92:0/3928377312 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.137+0000 7f8c3659c700 1 -- 172.21.15.92:0/3928377312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3819c170 con 0x7f8c38108750 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.138+0000 7f8c3659c700 1 --2- 172.21.15.92:0/3928377312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3819bc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c28003aa0 tx=0x7f8c28003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.138+0000 7f8c277fe700 1 -- 172.21.15.92:0/3928377312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c2800be50 con 0x7f8c38108750 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.138+0000 7f8c277fe700 1 -- 172.21.15.92:0/3928377312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c28004340 con 0x7f8c38108750 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.138+0000 7f8c277fe700 1 -- 172.21.15.92:0/3928377312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c2801a780 con 0x7f8c38108750 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.138+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3928377312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3819c400 con 0x7f8c38108750 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.138+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3928377312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3819f0f0 con 0x7f8c38108750 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.138+0000 7f8c277fe700 1 -- 172.21.15.92:0/3928377312 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8c28022020 con 0x7f8c38108750 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.138+0000 7f8c277fe700 1 -- 172.21.15.92:0/3928377312 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c28024d20 con 0x7f8c38108750 2024-03-19T01:42:56.891 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.139+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3928377312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c38063940 con 0x7f8c38108750 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.141+0000 7f8c277fe700 1 -- 172.21.15.92:0/3928377312 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8c280044f0 con 0x7f8c38108750 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.214+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3928377312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8c3819f2c0 con 0x7f8c38108750 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.214+0000 7f8c277fe700 1 -- 172.21.15.92:0/3928377312 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8c28011ac0 con 0x7f8c38108750 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.215+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3928377312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 msgr2=0x7f8c3819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.215+0000 7f8c3c99b700 1 --2- 172.21.15.92:0/3928377312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3819bc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c28003aa0 tx=0x7f8c28003e60 comp rx=0 tx=0).stop 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.215+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3928377312 shutdown_connections 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.215+0000 7f8c3c99b700 1 --2- 172.21.15.92:0/3928377312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c38108750 0x7f8c3819bc30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.215+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3928377312 >> 172.21.15.92:0/3928377312 conn(0x7f8c38102140 msgr2=0x7f8c38102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.216+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3928377312 shutdown_connections 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:56.216+0000 7f8c3c99b700 1 -- 172.21.15.92:0/3928377312 wait complete. 2024-03-19T01:42:56.892 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (2/15)... 2024-03-19T01:42:57.061 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:56 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:56.870+0000 7f5f26405200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:42:57.404 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:57 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:57.150+0000 7f5f26405200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:42:57.404 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:57 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:57.404+0000 7f5f26405200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:42:57.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:57 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:57.637+0000 7f5f26405200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:42:58.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:58 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:58.424+0000 7f5f26405200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:42:59.265 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:58 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:58.994+0000 7f5f26405200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:42:59.265 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:59.094+0000 7f5f26405200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:42:59.560 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:42:59.320+0000 7f5f26405200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:42:59.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: Activating manager daemon x 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: mgrmap e2: x(active, starting, since 0.00337046s) 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: Manager daemon x is now available 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' 2024-03-19T01:42:59.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:42:59 smithi092 ceph-mon[62574]: from='mgr.14100 172.21.15.92:0/509747761' entity='mgr.x' 2024-03-19T01:42:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:42:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-03-19T01:42:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:42:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T01:42:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T01:42:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T01:42:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T01:42:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T01:42:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T01:42:59.951 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T01:42:59.952 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T01:42:44.963671+0000", 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.297+0000 7f38ed085700 1 Processor -- start 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.297+0000 7f38ed085700 1 -- start start 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.297+0000 7f38ed085700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e8106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:59.953 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.297+0000 7f38ed085700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f38e8107570 con 0x7f38e8106b80 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.297+0000 7f38e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e8106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.297+0000 7f38e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e8106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41488/0 (socket says 172.21.15.92:41488) 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.297+0000 7f38e6d9d700 1 -- 172.21.15.92:0/3016554987 learned_addr learned my addr 172.21.15.92:0/3016554987 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.297+0000 7f38e6d9d700 1 -- 172.21.15.92:0/3016554987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e8107d80 con 0x7f38e8106b80 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.298+0000 7f38e6d9d700 1 --2- 172.21.15.92:0/3016554987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e8106fa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098a0 tx=0x7f38d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542fb10dbd64b500 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.298+0000 7f38e5d9b700 1 -- 172.21.15.92:0/3016554987 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d8004160 con 0x7f38e8106b80 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.298+0000 7f38e5d9b700 1 -- 172.21.15.92:0/3016554987 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38d8004300 con 0x7f38e8106b80 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.298+0000 7f38e5d9b700 1 -- 172.21.15.92:0/3016554987 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d8011b50 con 0x7f38e8106b80 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.299+0000 7f38ed085700 1 -- 172.21.15.92:0/3016554987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 msgr2=0x7f38e8106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.299+0000 7f38ed085700 1 --2- 172.21.15.92:0/3016554987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e8106fa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098a0 tx=0x7f38d8009bb0 comp rx=0 tx=0).stop 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.299+0000 7f38ed085700 1 -- 172.21.15.92:0/3016554987 shutdown_connections 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.299+0000 7f38ed085700 1 --2- 172.21.15.92:0/3016554987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e8106fa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:59.954 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.299+0000 7f38ed085700 1 -- 172.21.15.92:0/3016554987 >> 172.21.15.92:0/3016554987 conn(0x7f38e8102140 msgr2=0x7f38e8104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.299+0000 7f38ed085700 1 -- 172.21.15.92:0/3016554987 shutdown_connections 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.299+0000 7f38ed085700 1 -- 172.21.15.92:0/3016554987 wait complete. 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.299+0000 7f38ed085700 1 Processor -- start 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.300+0000 7f38ed085700 1 -- start start 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.300+0000 7f38ed085700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e807b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.300+0000 7f38ed085700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f38e810b130 con 0x7f38e8106b80 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.300+0000 7f38e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e807b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.300+0000 7f38e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e807b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41504/0 (socket says 172.21.15.92:41504) 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.300+0000 7f38e6d9d700 1 -- 172.21.15.92:0/628557218 learned_addr learned my addr 172.21.15.92:0/628557218 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.300+0000 7f38e6d9d700 1 -- 172.21.15.92:0/628557218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e819c7b0 con 0x7f38e8106b80 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.300+0000 7f38e6d9d700 1 --2- 172.21.15.92:0/628557218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e807b9d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009fb0 tx=0x7f38d8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.301+0000 7f38cffff700 1 -- 172.21.15.92:0/628557218 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d80038b0 con 0x7f38e8106b80 2024-03-19T01:42:59.955 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.301+0000 7f38cffff700 1 -- 172.21.15.92:0/628557218 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38d8011e50 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.301+0000 7f38cffff700 1 -- 172.21.15.92:0/628557218 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d8019550 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.301+0000 7f38ed085700 1 -- 172.21.15.92:0/628557218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e81a0dc0 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.301+0000 7f38ed085700 1 -- 172.21.15.92:0/628557218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e81a1160 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.301+0000 7f38cffff700 1 -- 172.21.15.92:0/628557218 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f38d8019be0 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.301+0000 7f38cffff700 1 -- 172.21.15.92:0/628557218 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f38d8024730 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.301+0000 7f38ed085700 1 -- 172.21.15.92:0/628557218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e8063940 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.305+0000 7f38cffff700 1 -- 172.21.15.92:0/628557218 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f38d8003a50 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.335+0000 7f38cffff700 1 -- 172.21.15.92:0/628557218 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 2) ==== 45862+0+0 (secure 0 0 0) 0x7f38d80249d0 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.385+0000 7f38ed085700 1 -- 172.21.15.92:0/628557218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f38e81a1940 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.388+0000 7f38cffff700 1 -- 172.21.15.92:0/628557218 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f38d8021730 con 0x7f38e8106b80 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.389+0000 7f38ed085700 1 -- 172.21.15.92:0/628557218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 msgr2=0x7f38e807b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.389+0000 7f38ed085700 1 --2- 172.21.15.92:0/628557218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e807b9d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009fb0 tx=0x7f38d8011bf0 comp rx=0 tx=0).stop 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.389+0000 7f38ed085700 1 -- 172.21.15.92:0/628557218 shutdown_connections 2024-03-19T01:42:59.956 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.389+0000 7f38ed085700 1 --2- 172.21.15.92:0/628557218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38e8106b80 0x7f38e807b9d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:42:59.957 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.389+0000 7f38ed085700 1 -- 172.21.15.92:0/628557218 >> 172.21.15.92:0/628557218 conn(0x7f38e8102140 msgr2=0x7f38e8102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:42:59.957 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.389+0000 7f38ed085700 1 -- 172.21.15.92:0/628557218 shutdown_connections 2024-03-19T01:42:59.957 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:42:59.389+0000 7f38ed085700 1 -- 172.21.15.92:0/628557218 wait complete. 2024-03-19T01:42:59.957 INFO:teuthology.orchestra.run.smithi092.stdout:mgr not available, waiting (3/15)... 2024-03-19T01:43:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:00 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/628557218' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T01:43:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:00 smithi092 ceph-mon[62574]: mgrmap e3: x(active, since 1.00676s) 2024-03-19T01:43:02.649 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:02 smithi092 ceph-mon[62574]: mgrmap e4: x(active, since 2s) 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "health": { 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 0 2024-03-19T01:43:03.246 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "a" 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-19T01:43:03.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-19T01:43:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-19T01:43:03.256 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-19T01:43:03.257 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "restful" 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ], 2024-03-19T01:43:03.258 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "modified": "2024-03-19T01:42:44.963671+0000", 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout }, 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd8af3700 1 Processor -- start 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd8af3700 1 -- start start 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd8af3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd4108090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd8af3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4108660 con 0x7f8cd4107c70 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd4108090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:03.259 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd4108090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41580/0 (socket says 172.21.15.92:41580) 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd259c700 1 -- 172.21.15.92:0/2880699900 learned_addr learned my addr 172.21.15.92:0/2880699900 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd259c700 1 -- 172.21.15.92:0/2880699900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd4108e70 con 0x7f8cd4107c70 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd259c700 1 --2- 172.21.15.92:0/2880699900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd4108090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80098a0 tx=0x7f8cc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fd3725766b73d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd159a700 1 -- 172.21.15.92:0/2880699900 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cc8004160 con 0x7f8cd4107c70 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd159a700 1 -- 172.21.15.92:0/2880699900 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cc8004300 con 0x7f8cd4107c70 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.366+0000 7f8cd159a700 1 -- 172.21.15.92:0/2880699900 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cc8011b50 con 0x7f8cd4107c70 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.367+0000 7f8cd8af3700 1 -- 172.21.15.92:0/2880699900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 msgr2=0x7f8cd4108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.367+0000 7f8cd8af3700 1 --2- 172.21.15.92:0/2880699900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd4108090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80098a0 tx=0x7f8cc8009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.367+0000 7f8cd8af3700 1 -- 172.21.15.92:0/2880699900 shutdown_connections 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.367+0000 7f8cd8af3700 1 --2- 172.21.15.92:0/2880699900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd4108090 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.367+0000 7f8cd8af3700 1 -- 172.21.15.92:0/2880699900 >> 172.21.15.92:0/2880699900 conn(0x7f8cd4103210 msgr2=0x7f8cd4105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.367+0000 7f8cd8af3700 1 -- 172.21.15.92:0/2880699900 shutdown_connections 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.367+0000 7f8cd8af3700 1 -- 172.21.15.92:0/2880699900 wait complete. 2024-03-19T01:43:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.368+0000 7f8cd8af3700 1 Processor -- start 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.368+0000 7f8cd8af3700 1 -- start start 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.368+0000 7f8cd8af3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.368+0000 7f8cd8af3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd410c220 con 0x7f8cd4107c70 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.368+0000 7f8cd259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.368+0000 7f8cd259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41590/0 (socket says 172.21.15.92:41590) 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.368+0000 7f8cd259c700 1 -- 172.21.15.92:0/3908171209 learned_addr learned my addr 172.21.15.92:0/3908171209 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.369+0000 7f8cd259c700 1 -- 172.21.15.92:0/3908171209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd419c200 con 0x7f8cd4107c70 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.369+0000 7f8cd259c700 1 --2- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd419bcc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009fb0 tx=0x7f8cc8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.369+0000 7f8cc37fe700 1 -- 172.21.15.92:0/3908171209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cc80038b0 con 0x7f8cd4107c70 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.369+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd419c490 con 0x7f8cd4107c70 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.369+0000 7f8cc37fe700 1 -- 172.21.15.92:0/3908171209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cc8011e50 con 0x7f8cd4107c70 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.369+0000 7f8cc37fe700 1 -- 172.21.15.92:0/3908171209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cc8019550 con 0x7f8cd4107c70 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.369+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd419f180 con 0x7f8cd4107c70 2024-03-19T01:43:03.261 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.370+0000 7f8cc37fe700 1 -- 172.21.15.92:0/3908171209 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f8cc8019850 con 0x7f8cd4107c70 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.370+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd4063940 con 0x7f8cd4107c70 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.370+0000 7f8cc37fe700 1 --2- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f8cbc038e80 0x7f8cbc03b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.370+0000 7f8cc37fe700 1 -- 172.21.15.92:0/3908171209 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8cc804d170 con 0x7f8cd4107c70 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.370+0000 7f8cd1d9b700 1 --2- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f8cbc038e80 0x7f8cbc03b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.370+0000 7f8cd1d9b700 1 --2- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f8cbc038e80 0x7f8cbc03b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4006fd0 tx=0x7f8cc4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.376+0000 7f8cc37fe700 1 -- 172.21.15.92:0/3908171209 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8cc800f2e0 con 0x7f8cd4107c70 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.648+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8cd419f350 con 0x7f8cd4107c70 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.649+0000 7f8cc37fe700 1 -- 172.21.15.92:0/3908171209 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8cd419f350 con 0x7f8cd4107c70 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f8cbc038e80 msgr2=0x7f8cbc03b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 --2- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f8cbc038e80 0x7f8cbc03b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4006fd0 tx=0x7f8cc4006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 msgr2=0x7f8cd419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 --2- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd419bcc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009fb0 tx=0x7f8cc8011bf0 comp rx=0 tx=0).stop 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 shutdown_connections 2024-03-19T01:43:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 --2- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f8cbc038e80 0x7f8cbc03b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:03.263 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 --2- 172.21.15.92:0/3908171209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd4107c70 0x7f8cd419bcc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:03.263 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 >> 172.21.15.92:0/3908171209 conn(0x7f8cd4103210 msgr2=0x7f8cd4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:03.263 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 shutdown_connections 2024-03-19T01:43:03.263 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:02.651+0000 7f8cd8af3700 1 -- 172.21.15.92:0/3908171209 wait complete. 2024-03-19T01:43:03.263 INFO:teuthology.orchestra.run.smithi092.stdout:mgr is available 2024-03-19T01:43:03.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:03 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3908171209' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-19T01:43:04.455 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:04 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3975080408' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-19T01:43:04.466 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:43:04.466 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [global] 2024-03-19T01:43:04.466 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout fsid = d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:43:04.466 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-19T01:43:04.466 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-19T01:43:04.466 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-19T01:43:04.466 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout [osd] 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.674+0000 7fe9fd8ea700 1 Processor -- start 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.674+0000 7fe9fd8ea700 1 -- start start 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.674+0000 7fe9fd8ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:04.467 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.674+0000 7fe9fd8ea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f807c890 con 0x7fe9f807de60 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.674+0000 7fe9f6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.674+0000 7fe9f6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48800/0 (socket says 172.21.15.92:48800) 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.674+0000 7fe9f6ffd700 1 -- 172.21.15.92:0/1206205755 learned_addr learned my addr 172.21.15.92:0/1206205755 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9f6ffd700 1 -- 172.21.15.92:0/1206205755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f807ca10 con 0x7fe9f807de60 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9f6ffd700 1 --2- 172.21.15.92:0/1206205755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f807c2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80098a0 tx=0x7fe9e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3128b21a08e77f3d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9f5ffb700 1 -- 172.21.15.92:0/1206205755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e8004160 con 0x7fe9f807de60 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9f5ffb700 1 -- 172.21.15.92:0/1206205755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9e8004300 con 0x7fe9f807de60 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9f5ffb700 1 -- 172.21.15.92:0/1206205755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e8011b50 con 0x7fe9f807de60 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/1206205755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 msgr2=0x7fe9f807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9fd8ea700 1 --2- 172.21.15.92:0/1206205755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f807c2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80098a0 tx=0x7fe9e8009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/1206205755 shutdown_connections 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9fd8ea700 1 --2- 172.21.15.92:0/1206205755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f807c2c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:04.468 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/1206205755 >> 172.21.15.92:0/1206205755 conn(0x7fe9f8103200 msgr2=0x7fe9f8105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/1206205755 shutdown_connections 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.675+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/1206205755 wait complete. 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.676+0000 7fe9fd8ea700 1 Processor -- start 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.676+0000 7fe9fd8ea700 1 -- start start 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.676+0000 7fe9fd8ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.676+0000 7fe9fd8ea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f810e240 con 0x7fe9f807de60 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.676+0000 7fe9f6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.676+0000 7fe9f6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f819de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48814/0 (socket says 172.21.15.92:48814) 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.677+0000 7fe9f6ffd700 1 -- 172.21.15.92:0/3975080408 learned_addr learned my addr 172.21.15.92:0/3975080408 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.677+0000 7fe9f6ffd700 1 -- 172.21.15.92:0/3975080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f819e3b0 con 0x7fe9f807de60 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.677+0000 7fe9f6ffd700 1 --2- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f819de70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009fb0 tx=0x7fe9e8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.677+0000 7fe9fc8e8700 1 -- 172.21.15.92:0/3975080408 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e80038b0 con 0x7fe9f807de60 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.677+0000 7fe9fc8e8700 1 -- 172.21.15.92:0/3975080408 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9e8011e50 con 0x7fe9f807de60 2024-03-19T01:43:04.469 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.677+0000 7fe9fc8e8700 1 -- 172.21.15.92:0/3975080408 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e80193c0 con 0x7fe9f807de60 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.677+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f819e640 con 0x7fe9f807de60 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.677+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f81a1330 con 0x7fe9f807de60 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.678+0000 7fe9fc8e8700 1 -- 172.21.15.92:0/3975080408 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7fe9e8019670 con 0x7fe9f807de60 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.678+0000 7fe9fc8e8700 1 --2- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7fe9e0038ed0 0x7fe9e003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.678+0000 7fe9fc8e8700 1 -- 172.21.15.92:0/3975080408 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe9e804d080 con 0x7fe9f807de60 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.678+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f8063940 con 0x7fe9f807de60 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.678+0000 7fe9f67fc700 1 --2- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7fe9e0038ed0 0x7fe9e003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.679+0000 7fe9f67fc700 1 --2- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7fe9e0038ed0 0x7fe9e003b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec006fd0 tx=0x7fe9ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.683+0000 7fe9fc8e8700 1 -- 172.21.15.92:0/3975080408 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe9e804cae0 con 0x7fe9f807de60 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.876+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe9f81a2a20 con 0x7fe9f807de60 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.877+0000 7fe9fc8e8700 1 -- 172.21.15.92:0/3975080408 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fe9e8032020 con 0x7fe9f807de60 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7fe9e0038ed0 msgr2=0x7fe9e003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:04.470 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 --2- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7fe9e0038ed0 0x7fe9e003b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec006fd0 tx=0x7fe9ec006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 msgr2=0x7fe9f819de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 --2- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f819de70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009fb0 tx=0x7fe9e8011bf0 comp rx=0 tx=0).stop 2024-03-19T01:43:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 shutdown_connections 2024-03-19T01:43:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 --2- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7fe9e0038ed0 0x7fe9e003b390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 --2- 172.21.15.92:0/3975080408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9f807de60 0x7fe9f819de70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 >> 172.21.15.92:0/3975080408 conn(0x7fe9f8103200 msgr2=0x7fe9f8103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 shutdown_connections 2024-03-19T01:43:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:03.879+0000 7fe9fd8ea700 1 -- 172.21.15.92:0/3975080408 wait complete. 2024-03-19T01:43:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling cephadm module... 2024-03-19T01:43:05.721 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:05 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1577811485' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-19T01:43:05.721 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:05 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: ignoring --setuser ceph since I am not root 2024-03-19T01:43:05.721 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:05 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: ignoring --setgroup ceph since I am not root 2024-03-19T01:43:06.005 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:05 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:05.955+0000 7f761404a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:43:06.012 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.872+0000 7ff9c7cbf700 1 Processor -- start 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.872+0000 7ff9c7cbf700 1 -- start start 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.872+0000 7ff9c7cbf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c0108600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.873+0000 7ff9c7cbf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0108bd0 con 0x7ff9c01081e0 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.873+0000 7ff9c5a5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c0108600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.873+0000 7ff9c5a5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c0108600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48830/0 (socket says 172.21.15.92:48830) 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.873+0000 7ff9c5a5b700 1 -- 172.21.15.92:0/1116812878 learned_addr learned my addr 172.21.15.92:0/1116812878 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.873+0000 7ff9c5a5b700 1 -- 172.21.15.92:0/1116812878 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c0105ce0 con 0x7ff9c01081e0 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.873+0000 7ff9c5a5b700 1 --2- 172.21.15.92:0/1116812878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c0108600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009e80 tx=0x7ff9bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150e8aac872c3c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.873+0000 7ff9c4a59700 1 -- 172.21.15.92:0/1116812878 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9bc017070 con 0x7ff9c01081e0 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.873+0000 7ff9c4a59700 1 -- 172.21.15.92:0/1116812878 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9bc00b8c0 con 0x7ff9c01081e0 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.873+0000 7ff9c4a59700 1 -- 172.21.15.92:0/1116812878 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9bc0045c0 con 0x7ff9c01081e0 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.874+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1116812878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 msgr2=0x7ff9c0108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:06.013 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.874+0000 7ff9c7cbf700 1 --2- 172.21.15.92:0/1116812878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c0108600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009e80 tx=0x7ff9bc00b330 comp rx=0 tx=0).stop 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.874+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1116812878 shutdown_connections 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.874+0000 7ff9c7cbf700 1 --2- 172.21.15.92:0/1116812878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c0108600 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.874+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1116812878 >> 172.21.15.92:0/1116812878 conn(0x7ff9c0101880 msgr2=0x7ff9c0103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.874+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1116812878 shutdown_connections 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.874+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1116812878 wait complete. 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.874+0000 7ff9c7cbf700 1 Processor -- start 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c7cbf700 1 -- start start 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c7cbf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c7cbf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0106cf0 con 0x7ff9c01081e0 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c5a5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c5a5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c019dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48834/0 (socket says 172.21.15.92:48834) 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c5a5b700 1 -- 172.21.15.92:0/1577811485 learned_addr learned my addr 172.21.15.92:0/1577811485 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:06.014 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c5a5b700 1 -- 172.21.15.92:0/1577811485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c019e2d0 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c5a5b700 1 --2- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c019dd90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009530 tx=0x7ff9bc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9b6ffd700 1 -- 172.21.15.92:0/1577811485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9bc020020 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9b6ffd700 1 -- 172.21.15.92:0/1577811485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9bc003b50 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c019e560 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9b6ffd700 1 -- 172.21.15.92:0/1577811485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9bc017040 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.875+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c01a1210 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.876+0000 7ff9b6ffd700 1 -- 172.21.15.92:0/1577811485 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7ff9bc023020 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.876+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c0063940 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.877+0000 7ff9b6ffd700 1 --2- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7ff9ac038ed0 0x7ff9ac03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.877+0000 7ff9b6ffd700 1 -- 172.21.15.92:0/1577811485 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9bc00f050 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.877+0000 7ff9c525a700 1 --2- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7ff9ac038ed0 0x7ff9ac03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.877+0000 7ff9c525a700 1 --2- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7ff9ac038ed0 0x7ff9ac03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0006fd0 tx=0x7ff9b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:04.882+0000 7ff9b6ffd700 1 -- 172.21.15.92:0/1577811485 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff9bc02a020 con 0x7ff9c01081e0 2024-03-19T01:43:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.134+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff9c0197370 con 0x7ff9c01081e0 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.393+0000 7ff9b6ffd700 1 -- 172.21.15.92:0/1577811485 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff9c0197370 con 0x7ff9c01081e0 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.395+0000 7ff9b6ffd700 1 -- 172.21.15.92:0/1577811485 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7ff9bc04ed10 con 0x7ff9c01081e0 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7ff9ac038ed0 msgr2=0x7ff9ac03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 --2- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7ff9ac038ed0 0x7ff9ac03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0006fd0 tx=0x7ff9b0006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 msgr2=0x7ff9c019dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 --2- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c019dd90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009530 tx=0x7ff9bc00bd50 comp rx=0 tx=0).stop 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 shutdown_connections 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 --2- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7ff9ac038ed0 0x7ff9ac03b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 --2- 172.21.15.92:0/1577811485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9c01081e0 0x7ff9c019dd90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 >> 172.21.15.92:0/1577811485 conn(0x7ff9c0101880 msgr2=0x7ff9c0194430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 shutdown_connections 2024-03-19T01:43:06.016 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:05.397+0000 7ff9c7cbf700 1 -- 172.21.15.92:0/1577811485 wait complete. 2024-03-19T01:43:06.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:06 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:06.055+0000 7f761404a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:43:06.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:06 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1577811485' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-19T01:43:06.637 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:06 smithi092 ceph-mon[62574]: mgrmap e5: x(active, since 6s) 2024-03-19T01:43:06.966 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:06 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:06.727+0000 7f761404a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:43:06.966 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:06 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:06.822+0000 7f761404a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:43:06.966 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:06 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:06.965+0000 7f761404a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:43:07.270 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:07 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:07.061+0000 7f761404a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:43:07.270 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:07 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:07.143+0000 7f761404a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:43:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-03-19T01:43:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-19T01:43:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-19T01:43:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-19T01:43:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-19T01:43:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-03-19T01:43:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.431+0000 7f2e6abfa700 1 Processor -- start 2024-03-19T01:43:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.431+0000 7f2e6abfa700 1 -- start start 2024-03-19T01:43:07.278 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.431+0000 7f2e6abfa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e64108610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.431+0000 7f2e6abfa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64108be0 con 0x7f2e641081f0 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.432+0000 7f2e68996700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e64108610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.432+0000 7f2e68996700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e64108610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48872/0 (socket says 172.21.15.92:48872) 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.432+0000 7f2e68996700 1 -- 172.21.15.92:0/3358172530 learned_addr learned my addr 172.21.15.92:0/3358172530 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.432+0000 7f2e68996700 1 -- 172.21.15.92:0/3358172530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e64105cf0 con 0x7f2e641081f0 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.432+0000 7f2e68996700 1 --2- 172.21.15.92:0/3358172530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e64108610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009ab0 tx=0x7f2e58009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9860669de062eb84 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.432+0000 7f2e637fe700 1 -- 172.21.15.92:0/3358172530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e58004160 con 0x7f2e641081f0 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.432+0000 7f2e637fe700 1 -- 172.21.15.92:0/3358172530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2e5800bca0 con 0x7f2e641081f0 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.432+0000 7f2e637fe700 1 -- 172.21.15.92:0/3358172530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e58003d60 con 0x7f2e641081f0 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.433+0000 7f2e6abfa700 1 -- 172.21.15.92:0/3358172530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 msgr2=0x7f2e64108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.433+0000 7f2e6abfa700 1 --2- 172.21.15.92:0/3358172530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e64108610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009ab0 tx=0x7f2e58009dc0 comp rx=0 tx=0).stop 2024-03-19T01:43:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.433+0000 7f2e6abfa700 1 -- 172.21.15.92:0/3358172530 shutdown_connections 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.433+0000 7f2e6abfa700 1 --2- 172.21.15.92:0/3358172530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e64108610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.433+0000 7f2e6abfa700 1 -- 172.21.15.92:0/3358172530 >> 172.21.15.92:0/3358172530 conn(0x7f2e641018d0 msgr2=0x7f2e64103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.433+0000 7f2e6abfa700 1 -- 172.21.15.92:0/3358172530 shutdown_connections 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.433+0000 7f2e6abfa700 1 -- 172.21.15.92:0/3358172530 wait complete. 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e6abfa700 1 Processor -- start 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e6abfa700 1 -- start start 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e6abfa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e6419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e6abfa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64106d00 con 0x7f2e641081f0 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e68996700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e6419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e68996700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e6419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48880/0 (socket says 172.21.15.92:48880) 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e68996700 1 -- 172.21.15.92:0/2685093123 learned_addr learned my addr 172.21.15.92:0/2685093123 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e68996700 1 -- 172.21.15.92:0/2685093123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6419e340 con 0x7f2e641081f0 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e68996700 1 --2- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e6419de00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2e58003aa0 tx=0x7f2e58003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:07.280 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.434+0000 7f2e61ffb700 1 -- 172.21.15.92:0/2685093123 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e5800be50 con 0x7f2e641081f0 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.435+0000 7f2e61ffb700 1 -- 172.21.15.92:0/2685093123 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2e58004340 con 0x7f2e641081f0 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.435+0000 7f2e61ffb700 1 -- 172.21.15.92:0/2685093123 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e5801a780 con 0x7f2e641081f0 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.435+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6419e5d0 con 0x7f2e641081f0 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.435+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e641a12c0 con 0x7f2e641081f0 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.436+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e64063940 con 0x7f2e641081f0 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.436+0000 7f2e61ffb700 1 -- 172.21.15.92:0/2685093123 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f2e58022020 con 0x7f2e641081f0 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.436+0000 7f2e61ffb700 1 --2- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f2e54038ed0 0x7f2e5403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.436+0000 7f2e61ffb700 1 -- 172.21.15.92:0/2685093123 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2e5804cb80 con 0x7f2e641081f0 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.436+0000 7f2e63fff700 1 -- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f2e54038ed0 msgr2=0x7f2e5403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1465441882 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.436+0000 7f2e63fff700 1 --2- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f2e54038ed0 0x7f2e5403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.441+0000 7f2e61ffb700 1 -- 172.21.15.92:0/2685093123 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2e5802c3a0 con 0x7f2e641081f0 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.637+0000 7f2e63fff700 1 -- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f2e54038ed0 msgr2=0x7f2e5403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1465441882 2024-03-19T01:43:07.281 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.637+0000 7f2e63fff700 1 --2- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f2e54038ed0 0x7f2e5403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.701+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f2e641a1490 con 0x7f2e641081f0 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.701+0000 7f2e61ffb700 1 -- 172.21.15.92:0/2685093123 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f2e641a1490 con 0x7f2e641081f0 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f2e54038ed0 msgr2=0x7f2e5403b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 --2- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f2e54038ed0 0x7f2e5403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 msgr2=0x7f2e6419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 --2- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e6419de00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2e58003aa0 tx=0x7f2e58003e60 comp rx=0 tx=0).stop 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 shutdown_connections 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 --2- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f2e54038ed0 0x7f2e5403b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 --2- 172.21.15.92:0/2685093123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e641081f0 0x7f2e6419de00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 >> 172.21.15.92:0/2685093123 conn(0x7f2e641018d0 msgr2=0x7f2e641944c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 shutdown_connections 2024-03-19T01:43:07.283 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:06.704+0000 7f2e6abfa700 1 -- 172.21.15.92:0/2685093123 wait complete. 2024-03-19T01:43:07.283 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-03-19T01:43:07.283 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 5... 2024-03-19T01:43:07.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:07 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2685093123' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-19T01:43:07.520 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:07 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:07.416+0000 7f761404a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:43:07.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:07 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:07.527+0000 7f761404a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:43:07.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:07 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:07.615+0000 7f761404a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:43:07.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:07 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:07.714+0000 7f761404a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:43:08.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:07 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:07.894+0000 7f761404a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:43:09.139 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:08 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:08.844+0000 7f761404a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:43:09.139 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:08 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:08.973+0000 7f761404a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:43:09.393 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:09 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:09.304+0000 7f761404a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:43:09.393 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:09 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:09.393+0000 7f761404a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:43:09.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:09 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:09.587+0000 7f761404a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:43:09.770 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:09 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:09.770+0000 7f761404a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:43:10.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:10 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:10.053+0000 7f761404a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:43:10.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:10 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:10.309+0000 7f761404a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:43:10.560 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:10 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:10.545+0000 7f761404a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:43:11.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:11 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:11.336+0000 7f761404a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:43:12.207 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:11 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:11.898+0000 7f761404a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:43:12.207 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:11 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:11.994+0000 7f761404a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:43:12.207 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:12.206+0000 7f761404a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:43:12.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: Active manager daemon x restarted 2024-03-19T01:43:12.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: Activating manager daemon x 2024-03-19T01:43:12.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: osdmap e2: 0 total, 0 up, 0 in 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: mgrmap e6: x(active, starting, since 0.00314064s) 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: Manager daemon x is now available 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:12 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:13.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:13 smithi092 ceph-mon[62574]: Found migration_current of "None". Setting to last migration. 2024-03-19T01:43:13.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:13 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:13.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:13 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T01:43:13.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:13 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T01:43:13.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:13 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:13.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:13 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:13.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:13 smithi092 ceph-mon[62574]: mgrmap e7: x(active, since 1.00533s) 2024-03-19T01:43:13.839 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-03-19T01:43:14.697 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-19T01:43:14.697 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-19T01:43:14.697 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-03-19T01:43:14.697 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.731+0000 7f376cea1700 1 Processor -- start 2024-03-19T01:43:14.697 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.731+0000 7f376cea1700 1 -- start start 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.731+0000 7f376cea1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f3768106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.731+0000 7f376cea1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3768106ce0 con 0x7f37681062f0 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.732+0000 7f376659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f3768106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.732+0000 7f376659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f3768106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48888/0 (socket says 172.21.15.92:48888) 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.732+0000 7f376659c700 1 -- 172.21.15.92:0/703275895 learned_addr learned my addr 172.21.15.92:0/703275895 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.732+0000 7f376659c700 1 -- 172.21.15.92:0/703275895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37681074f0 con 0x7f37681062f0 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.732+0000 7f376659c700 1 --2- 172.21.15.92:0/703275895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f3768106710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f375c00ac50 tx=0x7f375c00af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd1ee3c6b992969 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.732+0000 7f376559a700 1 -- 172.21.15.92:0/703275895 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f375c00b040 con 0x7f37681062f0 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.732+0000 7f376559a700 1 -- 172.21.15.92:0/703275895 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f375c0044f0 con 0x7f37681062f0 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.732+0000 7f376559a700 1 -- 172.21.15.92:0/703275895 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f375c0103f0 con 0x7f37681062f0 2024-03-19T01:43:14.698 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.733+0000 7f376cea1700 1 -- 172.21.15.92:0/703275895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 msgr2=0x7f3768106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.733+0000 7f376cea1700 1 --2- 172.21.15.92:0/703275895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f3768106710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f375c00ac50 tx=0x7f375c00af60 comp rx=0 tx=0).stop 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.733+0000 7f376cea1700 1 -- 172.21.15.92:0/703275895 shutdown_connections 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.733+0000 7f376cea1700 1 --2- 172.21.15.92:0/703275895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f3768106710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.733+0000 7f376cea1700 1 -- 172.21.15.92:0/703275895 >> 172.21.15.92:0/703275895 conn(0x7f37681018d0 msgr2=0x7f3768103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.733+0000 7f376cea1700 1 -- 172.21.15.92:0/703275895 shutdown_connections 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.733+0000 7f376cea1700 1 -- 172.21.15.92:0/703275895 wait complete. 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.734+0000 7f376cea1700 1 Processor -- start 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.734+0000 7f376cea1700 1 -- start start 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.734+0000 7f376cea1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f376819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.734+0000 7f376cea1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3768107c10 con 0x7f37681062f0 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.734+0000 7f376659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f376819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.734+0000 7f376659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f376819bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48900/0 (socket says 172.21.15.92:48900) 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.734+0000 7f376659c700 1 -- 172.21.15.92:0/1931437187 learned_addr learned my addr 172.21.15.92:0/1931437187 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:14.699 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.734+0000 7f376659c700 1 -- 172.21.15.92:0/1931437187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376819c190 con 0x7f37681062f0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.734+0000 7f376659c700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f376819bc50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f375c0046e0 tx=0x7f375c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.735+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f375c026070 con 0x7f37681062f0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.735+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f375c0036a0 con 0x7f37681062f0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.735+0000 7f376cea1700 1 -- 172.21.15.92:0/1931437187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376819c420 con 0x7f37681062f0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.735+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f375c021b50 con 0x7f37681062f0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.735+0000 7f376cea1700 1 -- 172.21.15.92:0/1931437187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376819f110 con 0x7f37681062f0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.736+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f375c029070 con 0x7f37681062f0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.736+0000 7f37577fe700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 0x7f375003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.736+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 --> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f375003baa0 con 0x7f3750038ed0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.736+0000 7f3765d9b700 1 -- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 msgr2=0x7f375003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1465441882 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.736+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f375c04d4a0 con 0x7f37681062f0 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.736+0000 7f3765d9b700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 0x7f375003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.937+0000 7f3765d9b700 1 -- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 msgr2=0x7f375003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1465441882 2024-03-19T01:43:14.700 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:07.937+0000 7f3765d9b700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 0x7f375003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:08.338+0000 7f3765d9b700 1 -- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 msgr2=0x7f375003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1465441882 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:08.338+0000 7f3765d9b700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 0x7f375003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:09.139+0000 7f3765d9b700 1 -- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 msgr2=0x7f375003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1465441882 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:09.139+0000 7f3765d9b700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 0x7f375003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:10.740+0000 7f3765d9b700 1 -- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 msgr2=0x7f375003b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1465441882 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:10.740+0000 7f3765d9b700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 0x7f375003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:12.222+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mgrmap(e 6) ==== 45873+0+0 (secure 0 0 0) 0x7f375c0213f0 con 0x7f37681062f0 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:12.222+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 msgr2=0x7f375003b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:12.222+0000 7f37577fe700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 0x7f375003b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.223+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7f375c02bce0 con 0x7f37681062f0 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.223+0000 7f37577fe700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f375003c7c0 0x7f375003ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.223+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f375c008e80 con 0x7f375003c7c0 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.223+0000 7f3765d9b700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f375003c7c0 0x7f375003ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.224+0000 7f3765d9b700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f375003c7c0 0x7f375003ebb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3758003a10 tx=0x7f3758009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:14.701 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.225+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 <== mgr.14120 v2:172.21.15.92:6800/1866676846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f375c008e80 con 0x7f375003c7c0 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 -- 172.21.15.92:0/1931437187 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3768063940 con 0x7f375003c7c0 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f37577fe700 1 -- 172.21.15.92:0/1931437187 <== mgr.14120 v2:172.21.15.92:6800/1866676846 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f3768063940 con 0x7f375003c7c0 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 -- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f375003c7c0 msgr2=0x7f375003ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f375003c7c0 0x7f375003ebb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3758003a10 tx=0x7f3758009290 comp rx=0 tx=0).stop 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 -- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 msgr2=0x7f376819bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f376819bc50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f375c0046e0 tx=0x7f375c003f50 comp rx=0 tx=0).stop 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 -- 172.21.15.92:0/1931437187 shutdown_connections 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f375003c7c0 0x7f375003ebb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37681062f0 0x7f376819bc50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:14.702 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 --2- 172.21.15.92:0/1931437187 >> [v2:172.21.15.92:6800/1465441882,v1:172.21.15.92:6801/1465441882] conn(0x7f3750038ed0 0x7f375003b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:14.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 -- 172.21.15.92:0/1931437187 >> 172.21.15.92:0/1931437187 conn(0x7f37681018d0 msgr2=0x7f3768192810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:14.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 -- 172.21.15.92:0/1931437187 shutdown_connections 2024-03-19T01:43:14.703 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:13.236+0000 7f376cea1700 1 -- 172.21.15.92:0/1931437187 wait complete. 2024-03-19T01:43:14.703 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 5 is available 2024-03-19T01:43:14.703 INFO:teuthology.orchestra.run.smithi092.stdout:Setting orchestrator backend to cephadm... 2024-03-19T01:43:14.704 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:14 smithi092 ceph-mon[62574]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-19T01:43:14.704 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:14 smithi092 ceph-mon[62574]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-19T01:43:14.704 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:14 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:15.058 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.296+0000 7fa20e488700 1 Processor -- start 2024-03-19T01:43:15.058 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.296+0000 7fa20e488700 1 -- start start 2024-03-19T01:43:15.058 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.296+0000 7fa20e488700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20807c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:15.058 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.296+0000 7fa20e488700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa20807c850 con 0x7fa20807de20 2024-03-19T01:43:15.058 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.296+0000 7fa207fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20807c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.296+0000 7fa207fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20807c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41784/0 (socket says 172.21.15.92:41784) 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.296+0000 7fa207fff700 1 -- 172.21.15.92:0/1388719771 learned_addr learned my addr 172.21.15.92:0/1388719771 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.297+0000 7fa207fff700 1 -- 172.21.15.92:0/1388719771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20807c9d0 con 0x7fa20807de20 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.297+0000 7fa207fff700 1 --2- 172.21.15.92:0/1388719771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20807c280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80098a0 tx=0x7fa1f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957397a65a066e79 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.297+0000 7fa206ffd700 1 -- 172.21.15.92:0/1388719771 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f8004160 con 0x7fa20807de20 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.297+0000 7fa206ffd700 1 -- 172.21.15.92:0/1388719771 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1f8004300 con 0x7fa20807de20 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.297+0000 7fa206ffd700 1 -- 172.21.15.92:0/1388719771 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f8011b50 con 0x7fa20807de20 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.297+0000 7fa20e488700 1 -- 172.21.15.92:0/1388719771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 msgr2=0x7fa20807c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.297+0000 7fa20e488700 1 --2- 172.21.15.92:0/1388719771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20807c280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80098a0 tx=0x7fa1f8009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.298+0000 7fa20e488700 1 -- 172.21.15.92:0/1388719771 shutdown_connections 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.298+0000 7fa20e488700 1 --2- 172.21.15.92:0/1388719771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20807c280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.298+0000 7fa20e488700 1 -- 172.21.15.92:0/1388719771 >> 172.21.15.92:0/1388719771 conn(0x7fa208103200 msgr2=0x7fa208105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.298+0000 7fa20e488700 1 -- 172.21.15.92:0/1388719771 shutdown_connections 2024-03-19T01:43:15.059 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.298+0000 7fa20e488700 1 -- 172.21.15.92:0/1388719771 wait complete. 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.298+0000 7fa20e488700 1 Processor -- start 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.298+0000 7fa20e488700 1 -- start start 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa20e488700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa20e488700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa20810e210 con 0x7fa20807de20 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa207fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa207fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20819de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41800/0 (socket says 172.21.15.92:41800) 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa207fff700 1 -- 172.21.15.92:0/3111735033 learned_addr learned my addr 172.21.15.92:0/3111735033 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa207fff700 1 -- 172.21.15.92:0/3111735033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20819e3b0 con 0x7fa20807de20 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa207fff700 1 --2- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20819de70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8011850 tx=0x7fa1f8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa2057fa700 1 -- 172.21.15.92:0/3111735033 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f8003af0 con 0x7fa20807de20 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20819e640 con 0x7fa20807de20 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa2057fa700 1 -- 172.21.15.92:0/3111735033 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1f8020070 con 0x7fa20807de20 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa2057fa700 1 -- 172.21.15.92:0/3111735033 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f80195a0 con 0x7fa20807de20 2024-03-19T01:43:15.060 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.299+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2081a1330 con 0x7fa20807de20 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.300+0000 7fa2057fa700 1 -- 172.21.15.92:0/3111735033 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fa1f80198a0 con 0x7fa20807de20 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.300+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa208063940 con 0x7fa20807de20 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.300+0000 7fa2057fa700 1 --2- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa1f0038f20 0x7fa1f003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.300+0000 7fa2057fa700 1 -- 172.21.15.92:0/3111735033 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa1f8011e00 con 0x7fa20807de20 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.300+0000 7fa2077fe700 1 --2- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa1f0038f20 0x7fa1f003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.301+0000 7fa2077fe700 1 --2- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa1f0038f20 0x7fa1f003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc006fd0 tx=0x7fa1fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.305+0000 7fa2057fa700 1 -- 172.21.15.92:0/3111735033 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa1f8019b90 con 0x7fa20807de20 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.520+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fa208104580 con 0x7fa1f0038f20 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.529+0000 7fa2057fa700 1 -- 172.21.15.92:0/3111735033 <== mgr.14120 v2:172.21.15.92:6800/1866676846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa208104580 con 0x7fa1f0038f20 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa1f0038f20 msgr2=0x7fa1f003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 --2- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa1f0038f20 0x7fa1f003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc006fd0 tx=0x7fa1fc006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 msgr2=0x7fa20819de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 --2- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20819de70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8011850 tx=0x7fa1f8011c10 comp rx=0 tx=0).stop 2024-03-19T01:43:15.061 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 shutdown_connections 2024-03-19T01:43:15.062 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 --2- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa1f0038f20 0x7fa1f003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:15.062 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 --2- 172.21.15.92:0/3111735033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa20807de20 0x7fa20819de70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:15.062 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 >> 172.21.15.92:0/3111735033 conn(0x7fa208103200 msgr2=0x7fa208103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:15.062 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 shutdown_connections 2024-03-19T01:43:15.062 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:14.532+0000 7fa20e488700 1 -- 172.21.15.92:0/3111735033 wait complete. 2024-03-19T01:43:15.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:15 smithi092 ceph-mon[62574]: [19/Mar/2024:01:43:13] ENGINE Bus STARTING 2024-03-19T01:43:15.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:15 smithi092 ceph-mon[62574]: [19/Mar/2024:01:43:13] ENGINE Serving on http://172.21.15.92:8765 2024-03-19T01:43:15.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:15 smithi092 ceph-mon[62574]: [19/Mar/2024:01:43:13] ENGINE Serving on https://172.21.15.92:7150 2024-03-19T01:43:15.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:15 smithi092 ceph-mon[62574]: [19/Mar/2024:01:43:13] ENGINE Bus STARTED 2024-03-19T01:43:15.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:15 smithi092 ceph-mon[62574]: mgrmap e8: x(active, since 2s) 2024-03-19T01:43:15.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:15 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:15.453 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:15 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:16.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:16 smithi092 ceph-mon[62574]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:16.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-19T01:43:16.330 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.476+0000 7f0967f60700 1 Processor -- start 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.476+0000 7f0967f60700 1 -- start start 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.476+0000 7f0967f60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096010aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.476+0000 7f0967f60700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f096007be30 con 0x7f0960108700 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.477+0000 7f0965cfc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096010aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.477+0000 7f0965cfc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096010aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41816/0 (socket says 172.21.15.92:41816) 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.477+0000 7f0965cfc700 1 -- 172.21.15.92:0/3021856866 learned_addr learned my addr 172.21.15.92:0/3021856866 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.477+0000 7f0965cfc700 1 -- 172.21.15.92:0/3021856866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096010b030 con 0x7f0960108700 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.477+0000 7f0965cfc700 1 --2- 172.21.15.92:0/3021856866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096010aaf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098a0 tx=0x7f095c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b577b9c7b8afd325 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.477+0000 7f0964cfa700 1 -- 172.21.15.92:0/3021856866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f095c004160 con 0x7f0960108700 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.477+0000 7f0964cfa700 1 -- 172.21.15.92:0/3021856866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f095c004300 con 0x7f0960108700 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.477+0000 7f0964cfa700 1 -- 172.21.15.92:0/3021856866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f095c011b50 con 0x7f0960108700 2024-03-19T01:43:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.478+0000 7f0967f60700 1 -- 172.21.15.92:0/3021856866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 msgr2=0x7f096010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.478+0000 7f0967f60700 1 --2- 172.21.15.92:0/3021856866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096010aaf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098a0 tx=0x7f095c009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.478+0000 7f0967f60700 1 -- 172.21.15.92:0/3021856866 shutdown_connections 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.478+0000 7f0967f60700 1 --2- 172.21.15.92:0/3021856866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096010aaf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.478+0000 7f0967f60700 1 -- 172.21.15.92:0/3021856866 >> 172.21.15.92:0/3021856866 conn(0x7f0960102150 msgr2=0x7f0960104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.478+0000 7f0967f60700 1 -- 172.21.15.92:0/3021856866 shutdown_connections 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.478+0000 7f0967f60700 1 -- 172.21.15.92:0/3021856866 wait complete. 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.478+0000 7f0967f60700 1 Processor -- start 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.478+0000 7f0967f60700 1 -- start start 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0967f60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096019bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0967f60700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f096010bff0 con 0x7f0960108700 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0965cfc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096019bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0965cfc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096019bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41830/0 (socket says 172.21.15.92:41830) 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0965cfc700 1 -- 172.21.15.92:0/2850937908 learned_addr learned my addr 172.21.15.92:0/2850937908 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:16.332 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0965cfc700 1 -- 172.21.15.92:0/2850937908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096019c180 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0965cfc700 1 --2- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096019bc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f095c009fb0 tx=0x7f095c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0956ffd700 1 -- 172.21.15.92:0/2850937908 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f095c0038b0 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0956ffd700 1 -- 172.21.15.92:0/2850937908 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f095c011e50 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096019c410 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0956ffd700 1 -- 172.21.15.92:0/2850937908 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f095c019420 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.479+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096019f100 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.480+0000 7f0956ffd700 1 -- 172.21.15.92:0/2850937908 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f095c019720 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.480+0000 7f0956ffd700 1 --2- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f094c038ed0 0x7f094c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.480+0000 7f0956ffd700 1 -- 172.21.15.92:0/2850937908 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f095c04d1a0 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.480+0000 7f09654fb700 1 --2- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f094c038ed0 0x7f094c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.480+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0960063940 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.481+0000 7f09654fb700 1 --2- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f094c038ed0 0x7f094c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.485+0000 7f0956ffd700 1 -- 172.21.15.92:0/2850937908 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f095c00bc70 con 0x7f0960108700 2024-03-19T01:43:16.333 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.680+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f0960103460 con 0x7f094c038ed0 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.681+0000 7f0956ffd700 1 -- 172.21.15.92:0/2850937908 <== mgr.14120 v2:172.21.15.92:6800/1866676846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f0960103460 con 0x7f094c038ed0 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f094c038ed0 msgr2=0x7f094c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 --2- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f094c038ed0 0x7f094c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 msgr2=0x7f096019bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 --2- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096019bc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f095c009fb0 tx=0x7f095c011bf0 comp rx=0 tx=0).stop 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 shutdown_connections 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 --2- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f094c038ed0 0x7f094c03b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 --2- 172.21.15.92:0/2850937908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0960108700 0x7f096019bc40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 >> 172.21.15.92:0/2850937908 conn(0x7f0960102150 msgr2=0x7f0960102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 shutdown_connections 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:15.683+0000 7f0967f60700 1 -- 172.21.15.92:0/2850937908 wait complete. 2024-03-19T01:43:16.334 INFO:teuthology.orchestra.run.smithi092.stdout:Generating ssh key... 2024-03-19T01:43:17.279 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: Generating public/private rsa key pair. 2024-03-19T01:43:17.279 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: Your identification has been saved in /tmp/tmpkgsamk50/key. 2024-03-19T01:43:17.279 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: Your public key has been saved in /tmp/tmpkgsamk50/key.pub. 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: The key fingerprint is: 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: SHA256:/c/wGlV4FxAkisTInNvwrgKBRp8fZ9C8DLvKggFJZ5Y ceph-d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: The key's randomart image is: 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: +---[RSA 3072]----+ 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: | + *. ..+o. | 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: | o E O.+. . . ..| 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: |+.= . X... . +| 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: |+o o + B . o.| 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: |o . . * S . . | 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: |.. o . . . | 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: |..o . . + | 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: |.. + . * | 2024-03-19T01:43:17.280 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: | . . ..+ | 2024-03-19T01:43:17.281 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: +----[SHA256]-----+ 2024-03-19T01:43:17.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-mon[62574]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:17.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:17.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:17 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.729+0000 7f16104c9700 1 Processor -- start 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.730+0000 7f16104c9700 1 -- start start 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.730+0000 7f16104c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f1608106720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.730+0000 7f16104c9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1608106cf0 con 0x7f1608106300 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.730+0000 7f160e265700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f1608106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.730+0000 7f160e265700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f1608106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41842/0 (socket says 172.21.15.92:41842) 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.730+0000 7f160e265700 1 -- 172.21.15.92:0/1198186645 learned_addr learned my addr 172.21.15.92:0/1198186645 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.730+0000 7f160e265700 1 -- 172.21.15.92:0/1198186645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1608107500 con 0x7f1608106300 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f160e265700 1 --2- 172.21.15.92:0/1198186645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f1608106720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1600009e80 tx=0x7f160000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae187e92dc03069 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f160d263700 1 -- 172.21.15.92:0/1198186645 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1600017070 con 0x7f1608106300 2024-03-19T01:43:17.663 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f160d263700 1 -- 172.21.15.92:0/1198186645 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f160000b8c0 con 0x7f1608106300 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f160d263700 1 -- 172.21.15.92:0/1198186645 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16000045c0 con 0x7f1608106300 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f16104c9700 1 -- 172.21.15.92:0/1198186645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 msgr2=0x7f1608106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f16104c9700 1 --2- 172.21.15.92:0/1198186645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f1608106720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1600009e80 tx=0x7f160000b330 comp rx=0 tx=0).stop 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f16104c9700 1 -- 172.21.15.92:0/1198186645 shutdown_connections 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f16104c9700 1 --2- 172.21.15.92:0/1198186645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f1608106720 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f16104c9700 1 -- 172.21.15.92:0/1198186645 >> 172.21.15.92:0/1198186645 conn(0x7f16081018c0 msgr2=0x7f1608103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f16104c9700 1 -- 172.21.15.92:0/1198186645 shutdown_connections 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.731+0000 7f16104c9700 1 -- 172.21.15.92:0/1198186645 wait complete. 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.732+0000 7f16104c9700 1 Processor -- start 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.732+0000 7f16104c9700 1 -- start start 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.732+0000 7f16104c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f160810b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.732+0000 7f16104c9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1608107c20 con 0x7f1608106300 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.732+0000 7f160e265700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f160810b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:17.664 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.732+0000 7f160e265700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f160810b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41856/0 (socket says 172.21.15.92:41856) 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.732+0000 7f160e265700 1 -- 172.21.15.92:0/3555746836 learned_addr learned my addr 172.21.15.92:0/3555746836 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.733+0000 7f160e265700 1 -- 172.21.15.92:0/3555746836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160819e970 con 0x7f1608106300 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.733+0000 7f160e265700 1 --2- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f160810b160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1600004210 tx=0x7f160000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.733+0000 7f16077fe700 1 -- 172.21.15.92:0/3555746836 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1600020020 con 0x7f1608106300 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.733+0000 7f16077fe700 1 -- 172.21.15.92:0/3555746836 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1600003b50 con 0x7f1608106300 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.733+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160819eb40 con 0x7f1608106300 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.733+0000 7f16077fe700 1 -- 172.21.15.92:0/3555746836 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1600017040 con 0x7f1608106300 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.733+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160819eee0 con 0x7f1608106300 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.734+0000 7f16077fe700 1 -- 172.21.15.92:0/3555746836 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f1600023020 con 0x7f1608106300 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.734+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1608063940 con 0x7f1608106300 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.734+0000 7f16077fe700 1 --2- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f15f0038f70 0x7f15f003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.734+0000 7f16077fe700 1 -- 172.21.15.92:0/3555746836 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f160000f050 con 0x7f1608106300 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.734+0000 7f160da64700 1 --2- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f15f0038f70 0x7f15f003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:17.665 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.734+0000 7f160da64700 1 --2- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f15f0038f70 0x7f15f003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f15fc006fd0 tx=0x7f15fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.739+0000 7f16077fe700 1 -- 172.21.15.92:0/3555746836 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f160002a020 con 0x7f1608106300 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:16.933+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f16081027d0 con 0x7f15f0038f70 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.042+0000 7f16077fe700 1 -- 172.21.15.92:0/3555746836 <== mgr.14120 v2:172.21.15.92:6800/1866676846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f16081027d0 con 0x7f15f0038f70 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f15f0038f70 msgr2=0x7f15f003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 --2- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f15f0038f70 0x7f15f003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f15fc006fd0 tx=0x7f15fc006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 msgr2=0x7f160810b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 --2- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f160810b160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1600004210 tx=0x7f160000bd50 comp rx=0 tx=0).stop 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 shutdown_connections 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 --2- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f15f0038f70 0x7f15f003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 --2- 172.21.15.92:0/3555746836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1608106300 0x7f160810b160 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 >> 172.21.15.92:0/3555746836 conn(0x7f16081018c0 msgr2=0x7f16081020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 shutdown_connections 2024-03-19T01:43:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:17.044+0000 7f16104c9700 1 -- 172.21.15.92:0/3555746836 wait complete. 2024-03-19T01:43:18.552 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:18 smithi092 ceph-mon[62574]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:18.552 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:18 smithi092 ceph-mon[62574]: Generating ssh key... 2024-03-19T01:43:18.861 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEk/umEnrSLoAXRE8bb9OUNzE09wfce/czVvxSzfTikpPcyy9vsloqk3OUHgJhV5n6nhoI5TVkWzVqwJUlubn6U6JFJIFwoSzUc99e5tHCMcFNqfPxBPDs80n1KzCL3cJbCatFuNCQ7Yz7TMmC/1cOdrXButX86U7bgs2NJqPaxVaDGf5JlLwUj3ViaD2EcSba5cJFyYt7cj4/i5xkFC9SEv5uOGf6HLEFXKuhxgN9KwUOEg6lqMIl9t9QP2aLkfgrx63Zc81eSBo2P1/Ic0FbrD38YMZjzAUzkZa6hGJ7ofBC3PZ49Kai6Qo5nO/QyhrX+YmNE7TwstGdxGduNF1vMTDHxKIsdtV9Bd1VqvlatWR8mWAG0khY9fsRhRw4PAOzWxjIAC0ybeYdsEY9DESvGDfaDK/TO4jBq73ra9zJnCHdiw6FGdSl02O3iiZLcT1v4Nr7CJaPrzkxblhkhxn/nRQ1xty2LF/BvZ6uTu+9WcWDShJxsGa+gfZv9/mO4tc= ceph-d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:43:18.861 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.045+0000 7f4373dab700 1 Processor -- start 2024-03-19T01:43:18.861 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.045+0000 7f4373dab700 1 -- start start 2024-03-19T01:43:18.861 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4373dab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:18.861 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4373dab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f436c107590 con 0x7f436c106ba0 2024-03-19T01:43:18.861 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4371b47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:18.861 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4371b47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41870/0 (socket says 172.21.15.92:41870) 2024-03-19T01:43:18.861 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4371b47700 1 -- 172.21.15.92:0/2980930172 learned_addr learned my addr 172.21.15.92:0/2980930172 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:18.861 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4371b47700 1 -- 172.21.15.92:0/2980930172 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c107da0 con 0x7f436c106ba0 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4371b47700 1 --2- 172.21.15.92:0/2980930172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c106fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f435c0098a0 tx=0x7f435c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa89ebc4ffab6e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4370b45700 1 -- 172.21.15.92:0/2980930172 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f435c004160 con 0x7f436c106ba0 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4370b45700 1 -- 172.21.15.92:0/2980930172 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f435c004300 con 0x7f436c106ba0 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.046+0000 7f4370b45700 1 -- 172.21.15.92:0/2980930172 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f435c011b50 con 0x7f436c106ba0 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.047+0000 7f4373dab700 1 -- 172.21.15.92:0/2980930172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 msgr2=0x7f436c106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.047+0000 7f4373dab700 1 --2- 172.21.15.92:0/2980930172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c106fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f435c0098a0 tx=0x7f435c009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.047+0000 7f4373dab700 1 -- 172.21.15.92:0/2980930172 shutdown_connections 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.047+0000 7f4373dab700 1 --2- 172.21.15.92:0/2980930172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c106fc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.047+0000 7f4373dab700 1 -- 172.21.15.92:0/2980930172 >> 172.21.15.92:0/2980930172 conn(0x7f436c102140 msgr2=0x7f436c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.047+0000 7f4373dab700 1 -- 172.21.15.92:0/2980930172 shutdown_connections 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.047+0000 7f4373dab700 1 -- 172.21.15.92:0/2980930172 wait complete. 2024-03-19T01:43:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.047+0000 7f4373dab700 1 Processor -- start 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.047+0000 7f4373dab700 1 -- start start 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4373dab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c07b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4373dab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f436c10b150 con 0x7f436c106ba0 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4371b47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c07b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4371b47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c07b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41882/0 (socket says 172.21.15.92:41882) 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4371b47700 1 -- 172.21.15.92:0/1119534514 learned_addr learned my addr 172.21.15.92:0/1119534514 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4371b47700 1 -- 172.21.15.92:0/1119534514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c19e9b0 con 0x7f436c106ba0 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4371b47700 1 --2- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c07b9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f435c011850 tx=0x7f435c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4362ffd700 1 -- 172.21.15.92:0/1119534514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f435c003af0 con 0x7f436c106ba0 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4362ffd700 1 -- 172.21.15.92:0/1119534514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f435c020070 con 0x7f436c106ba0 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4362ffd700 1 -- 172.21.15.92:0/1119534514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f435c0195a0 con 0x7f436c106ba0 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436c19eb80 con 0x7f436c106ba0 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.048+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436c19ef20 con 0x7f436c106ba0 2024-03-19T01:43:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.049+0000 7f4362ffd700 1 -- 172.21.15.92:0/1119534514 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f435c019850 con 0x7f436c106ba0 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.049+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436c063940 con 0x7f436c106ba0 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.049+0000 7f4362ffd700 1 --2- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f4358038f70 0x7f435803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.049+0000 7f4362ffd700 1 -- 172.21.15.92:0/1119534514 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f435c011e00 con 0x7f436c106ba0 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.049+0000 7f4371346700 1 --2- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f4358038f70 0x7f435803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.050+0000 7f4371346700 1 --2- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f4358038f70 0x7f435803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4368006fd0 tx=0x7f4368006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.054+0000 7f4362ffd700 1 -- 172.21.15.92:0/1119534514 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f435c023370 con 0x7f436c106ba0 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.252+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f436c1034a0 con 0x7f4358038f70 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.253+0000 7f4362ffd700 1 -- 172.21.15.92:0/1119534514 <== mgr.14120 v2:172.21.15.92:6800/1866676846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f436c1034a0 con 0x7f4358038f70 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f4358038f70 msgr2=0x7f435803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 --2- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f4358038f70 0x7f435803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4368006fd0 tx=0x7f4368006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 msgr2=0x7f436c07b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:18.864 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 --2- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c07b9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f435c011850 tx=0x7f435c011c10 comp rx=0 tx=0).stop 2024-03-19T01:43:18.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 shutdown_connections 2024-03-19T01:43:18.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 --2- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f4358038f70 0x7f435803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:18.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 --2- 172.21.15.92:0/1119534514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f436c106ba0 0x7f436c07b9d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:18.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 >> 172.21.15.92:0/1119534514 conn(0x7f436c102140 msgr2=0x7f436c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:18.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 shutdown_connections 2024-03-19T01:43:18.865 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:18.255+0000 7f4373dab700 1 -- 172.21.15.92:0/1119534514 wait complete. 2024-03-19T01:43:18.865 INFO:teuthology.orchestra.run.smithi092.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-19T01:43:18.865 INFO:teuthology.orchestra.run.smithi092.stdout:Adding key to root@localhost authorized_keys... 2024-03-19T01:43:18.865 INFO:teuthology.orchestra.run.smithi092.stdout:Adding host smithi092... 2024-03-19T01:43:19.495 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:19 smithi092 ceph-mon[62574]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:20 smithi092 ceph-mon[62574]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "addr": "172.21.15.92", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:20 smithi092 ceph-mon[62574]: Deploying cephadm binary to smithi092 2024-03-19T01:43:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Added host 'smithi092' with addr '172.21.15.92' 2024-03-19T01:43:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.288+0000 7f8915507700 1 Processor -- start 2024-03-19T01:43:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.288+0000 7f8915507700 1 -- start start 2024-03-19T01:43:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.288+0000 7f8915507700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891010ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.288+0000 7f8915507700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f891007bdf0 con 0x7f8910108720 2024-03-19T01:43:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.288+0000 7f890effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891010ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.288+0000 7f890effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891010ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41894/0 (socket says 172.21.15.92:41894) 2024-03-19T01:43:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.288+0000 7f890effd700 1 -- 172.21.15.92:0/2477762984 learned_addr learned my addr 172.21.15.92:0/2477762984 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.288+0000 7f890effd700 1 -- 172.21.15.92:0/2477762984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891010b050 con 0x7f8910108720 2024-03-19T01:43:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.288+0000 7f890effd700 1 --2- 172.21.15.92:0/2477762984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891010ab10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89000098a0 tx=0x7f8900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1f368023eb4b3c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f890dffb700 1 -- 172.21.15.92:0/2477762984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900004160 con 0x7f8910108720 2024-03-19T01:43:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f890dffb700 1 -- 172.21.15.92:0/2477762984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8900004300 con 0x7f8910108720 2024-03-19T01:43:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f890dffb700 1 -- 172.21.15.92:0/2477762984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900011b50 con 0x7f8910108720 2024-03-19T01:43:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f8915507700 1 -- 172.21.15.92:0/2477762984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 msgr2=0x7f891010ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f8915507700 1 --2- 172.21.15.92:0/2477762984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891010ab10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89000098a0 tx=0x7f8900009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f8915507700 1 -- 172.21.15.92:0/2477762984 shutdown_connections 2024-03-19T01:43:24.286 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f8915507700 1 --2- 172.21.15.92:0/2477762984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891010ab10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f8915507700 1 -- 172.21.15.92:0/2477762984 >> 172.21.15.92:0/2477762984 conn(0x7f8910102130 msgr2=0x7f8910104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f8915507700 1 -- 172.21.15.92:0/2477762984 shutdown_connections 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.289+0000 7f8915507700 1 -- 172.21.15.92:0/2477762984 wait complete. 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.290+0000 7f8915507700 1 Processor -- start 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.290+0000 7f8915507700 1 -- start start 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.290+0000 7f8915507700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.290+0000 7f8915507700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f891010c000 con 0x7f8910108720 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.290+0000 7f890effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.290+0000 7f890effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41906/0 (socket says 172.21.15.92:41906) 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.290+0000 7f890effd700 1 -- 172.21.15.92:0/675630626 learned_addr learned my addr 172.21.15.92:0/675630626 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.291+0000 7f890effd700 1 -- 172.21.15.92:0/675630626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891019c190 con 0x7f8910108720 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.291+0000 7f890effd700 1 --2- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891019bc50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8900011b60 tx=0x7f8900011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:24.289 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.291+0000 7f88f7fff700 1 -- 172.21.15.92:0/675630626 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900003970 con 0x7f8910108720 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.291+0000 7f88f7fff700 1 -- 172.21.15.92:0/675630626 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8900003b10 con 0x7f8910108720 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.291+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891019c420 con 0x7f8910108720 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.291+0000 7f88f7fff700 1 -- 172.21.15.92:0/675630626 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900024610 con 0x7f8910108720 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.291+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891019f110 con 0x7f8910108720 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.292+0000 7f88f7fff700 1 -- 172.21.15.92:0/675630626 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f89000248c0 con 0x7f8910108720 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.292+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8910063940 con 0x7f8910108720 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.292+0000 7f88f7fff700 1 --2- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f88f8038f70 0x7f88f803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.292+0000 7f88f7fff700 1 -- 172.21.15.92:0/675630626 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8900011e00 con 0x7f8910108720 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.292+0000 7f890e7fc700 1 --2- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f88f8038f70 0x7f88f803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.292+0000 7f890e7fc700 1 --2- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f88f8038f70 0x7f88f803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8904006fd0 tx=0x7f8904006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.297+0000 7f88f7fff700 1 -- 172.21.15.92:0/675630626 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f890000ba00 con 0x7f8910108720 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:19.495+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi092", "addr": "172.21.15.92", "target": ["mon-mgr", ""]}) -- 0x7f8910103420 con 0x7f88f8038f70 2024-03-19T01:43:24.290 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.656+0000 7f88f7fff700 1 -- 172.21.15.92:0/675630626 <== mgr.14120 v2:172.21.15.92:6800/1866676846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8910103420 con 0x7f88f8038f70 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f88f8038f70 msgr2=0x7f88f803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 --2- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f88f8038f70 0x7f88f803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8904006fd0 tx=0x7f8904006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 msgr2=0x7f891019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 --2- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891019bc50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8900011b60 tx=0x7f8900011c40 comp rx=0 tx=0).stop 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 shutdown_connections 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 --2- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f88f8038f70 0x7f88f803b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 --2- 172.21.15.92:0/675630626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8910108720 0x7f891019bc50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 >> 172.21.15.92:0/675630626 conn(0x7f8910102130 msgr2=0x7f8910102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 shutdown_connections 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:23.659+0000 7f8915507700 1 -- 172.21.15.92:0/675630626 wait complete. 2024-03-19T01:43:24.291 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying unmanaged mon service... 2024-03-19T01:43:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:24 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:24 smithi092 ceph-mon[62574]: Added host smithi092 2024-03-19T01:43:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:24 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-19T01:43:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.658+0000 7f1349982700 1 Processor -- start 2024-03-19T01:43:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.658+0000 7f1349982700 1 -- start start 2024-03-19T01:43:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.658+0000 7f1349982700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f13441080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.658+0000 7f1349982700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1344108690 con 0x7f1344107ca0 2024-03-19T01:43:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.658+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f13441080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.658+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f13441080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57158/0 (socket says 172.21.15.92:57158) 2024-03-19T01:43:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.658+0000 7f1342ffd700 1 -- 172.21.15.92:0/2484346059 learned_addr learned my addr 172.21.15.92:0/2484346059 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.658+0000 7f1342ffd700 1 -- 172.21.15.92:0/2484346059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1344108ea0 con 0x7f1344107ca0 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.658+0000 7f1342ffd700 1 --2- 172.21.15.92:0/2484346059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f13441080c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f13340098a0 tx=0x7f1334009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232e84307917aee8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1341ffb700 1 -- 172.21.15.92:0/2484346059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1334004160 con 0x7f1344107ca0 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1341ffb700 1 -- 172.21.15.92:0/2484346059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1334004300 con 0x7f1344107ca0 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1341ffb700 1 -- 172.21.15.92:0/2484346059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1334011b50 con 0x7f1344107ca0 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1349982700 1 -- 172.21.15.92:0/2484346059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 msgr2=0x7f13441080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1349982700 1 --2- 172.21.15.92:0/2484346059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f13441080c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f13340098a0 tx=0x7f1334009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1349982700 1 -- 172.21.15.92:0/2484346059 shutdown_connections 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1349982700 1 --2- 172.21.15.92:0/2484346059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f13441080c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1349982700 1 -- 172.21.15.92:0/2484346059 >> 172.21.15.92:0/2484346059 conn(0x7f1344103280 msgr2=0x7f13441056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1349982700 1 -- 172.21.15.92:0/2484346059 shutdown_connections 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.659+0000 7f1349982700 1 -- 172.21.15.92:0/2484346059 wait complete. 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.660+0000 7f1349982700 1 Processor -- start 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.660+0000 7f1349982700 1 -- start start 2024-03-19T01:43:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.660+0000 7f1349982700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f134419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.660+0000 7f1349982700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f134410c250 con 0x7f1344107ca0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.660+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f134419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.660+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f134419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57160/0 (socket says 172.21.15.92:57160) 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.660+0000 7f1342ffd700 1 -- 172.21.15.92:0/3862470115 learned_addr learned my addr 172.21.15.92:0/3862470115 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.660+0000 7f1342ffd700 1 -- 172.21.15.92:0/3862470115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134419c1a0 con 0x7f1344107ca0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.661+0000 7f1342ffd700 1 --2- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f134419bc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1334011850 tx=0x7f1334011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.661+0000 7f1348980700 1 -- 172.21.15.92:0/3862470115 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1334003af0 con 0x7f1344107ca0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.661+0000 7f1348980700 1 -- 172.21.15.92:0/3862470115 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1334020070 con 0x7f1344107ca0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.661+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134419c430 con 0x7f1344107ca0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.661+0000 7f1348980700 1 -- 172.21.15.92:0/3862470115 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13340195a0 con 0x7f1344107ca0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.661+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134419f120 con 0x7f1344107ca0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.662+0000 7f1348980700 1 -- 172.21.15.92:0/3862470115 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f13340198a0 con 0x7f1344107ca0 2024-03-19T01:43:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.662+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1344063940 con 0x7f1344107ca0 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.662+0000 7f1348980700 1 --2- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f132c038f20 0x7f132c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.662+0000 7f1348980700 1 -- 172.21.15.92:0/3862470115 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1334011e00 con 0x7f1344107ca0 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.662+0000 7f13427fc700 1 --2- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f132c038f20 0x7f132c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.663+0000 7f13427fc700 1 --2- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f132c038f20 0x7f132c03b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1338006fd0 tx=0x7f1338006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.667+0000 7f1348980700 1 -- 172.21.15.92:0/3862470115 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1334003c90 con 0x7f1344107ca0 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.886+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f13441045b0 con 0x7f132c038f20 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.894+0000 7f1348980700 1 -- 172.21.15.92:0/3862470115 <== mgr.14120 v2:172.21.15.92:6800/1866676846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f13441045b0 con 0x7f132c038f20 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.896+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f132c038f20 msgr2=0x7f132c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.896+0000 7f1349982700 1 --2- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f132c038f20 0x7f132c03b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1338006fd0 tx=0x7f1338006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.896+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 msgr2=0x7f134419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.896+0000 7f1349982700 1 --2- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f134419bc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1334011850 tx=0x7f1334011c10 comp rx=0 tx=0).stop 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.896+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 shutdown_connections 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.896+0000 7f1349982700 1 --2- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f132c038f20 0x7f132c03b3e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.896+0000 7f1349982700 1 --2- 172.21.15.92:0/3862470115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1344107ca0 0x7f134419bc60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:25.684 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.896+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 >> 172.21.15.92:0/3862470115 conn(0x7f1344103280 msgr2=0x7f1344103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:25.684 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.897+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 shutdown_connections 2024-03-19T01:43:25.684 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:24.897+0000 7f1349982700 1 -- 172.21.15.92:0/3862470115 wait complete. 2024-03-19T01:43:25.684 INFO:teuthology.orchestra.run.smithi092.stdout:Deploying unmanaged mgr service... 2024-03-19T01:43:25.964 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:25 smithi092 ceph-mon[62574]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:25.964 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:25 smithi092 ceph-mon[62574]: Saving service mon spec with placement count:5 2024-03-19T01:43:25.964 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:25 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:27.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:27 smithi092 ceph-mon[62574]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:27.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:27 smithi092 ceph-mon[62574]: Saving service mgr spec with placement count:2 2024-03-19T01:43:27.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:27 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:27.673 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.064+0000 7f93b8245700 1 Processor -- start 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.065+0000 7f93b8245700 1 -- start start 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.065+0000 7f93b8245700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b010aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.065+0000 7f93b8245700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f93b007be30 con 0x7f93b01086f0 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.065+0000 7f93b5fe1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b010aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.065+0000 7f93b5fe1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b010aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57176/0 (socket says 172.21.15.92:57176) 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.065+0000 7f93b5fe1700 1 -- 172.21.15.92:0/4196982224 learned_addr learned my addr 172.21.15.92:0/4196982224 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.065+0000 7f93b5fe1700 1 -- 172.21.15.92:0/4196982224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b010b020 con 0x7f93b01086f0 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.066+0000 7f93b5fe1700 1 --2- 172.21.15.92:0/4196982224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b010aae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0098a0 tx=0x7f93ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c25a0e5043661c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.066+0000 7f93b4fdf700 1 -- 172.21.15.92:0/4196982224 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93ac004160 con 0x7f93b01086f0 2024-03-19T01:43:27.674 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.066+0000 7f93b4fdf700 1 -- 172.21.15.92:0/4196982224 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93ac004300 con 0x7f93b01086f0 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.066+0000 7f93b4fdf700 1 -- 172.21.15.92:0/4196982224 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93ac011b50 con 0x7f93b01086f0 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 -- 172.21.15.92:0/4196982224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 msgr2=0x7f93b010aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 --2- 172.21.15.92:0/4196982224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b010aae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0098a0 tx=0x7f93ac009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 -- 172.21.15.92:0/4196982224 shutdown_connections 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 --2- 172.21.15.92:0/4196982224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b010aae0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 -- 172.21.15.92:0/4196982224 >> 172.21.15.92:0/4196982224 conn(0x7f93b0102140 msgr2=0x7f93b0104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 -- 172.21.15.92:0/4196982224 shutdown_connections 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 -- 172.21.15.92:0/4196982224 wait complete. 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 Processor -- start 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 -- start start 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.067+0000 7f93b8245700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010bfe0 con 0x7f93b01086f0 2024-03-19T01:43:27.675 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93b5fe1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93b5fe1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57188/0 (socket says 172.21.15.92:57188) 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93b5fe1700 1 -- 172.21.15.92:0/1218706721 learned_addr learned my addr 172.21.15.92:0/1218706721 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93b5fe1700 1 -- 172.21.15.92:0/1218706721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b019c150 con 0x7f93b01086f0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93b5fe1700 1 --2- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b019bc10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009fb0 tx=0x7f93ac011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93a6ffd700 1 -- 172.21.15.92:0/1218706721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93ac0038b0 con 0x7f93b01086f0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93a6ffd700 1 -- 172.21.15.92:0/1218706721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93ac011e50 con 0x7f93b01086f0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b019c3e0 con 0x7f93b01086f0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93a6ffd700 1 -- 172.21.15.92:0/1218706721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93ac019420 con 0x7f93b01086f0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.068+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b019f0d0 con 0x7f93b01086f0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.069+0000 7f93a6ffd700 1 -- 172.21.15.92:0/1218706721 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f93ac0197a0 con 0x7f93b01086f0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.069+0000 7f93a6ffd700 1 --2- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f939c038ed0 0x7f939c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.069+0000 7f93a6ffd700 1 -- 172.21.15.92:0/1218706721 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f93ac04e160 con 0x7f93b01086f0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.069+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b019c8c0 con 0x7f93b01086f0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.069+0000 7f93b57e0700 1 --2- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f939c038ed0 0x7f939c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:27.676 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.070+0000 7f93b57e0700 1 --2- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f939c038ed0 0x7f939c03b390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93a0006fd0 tx=0x7f93a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.075+0000 7f93a6ffd700 1 -- 172.21.15.92:0/1218706721 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f93b019c8c0 con 0x7f93b01086f0 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.295+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f93b01033e0 con 0x7f939c038ed0 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.302+0000 7f93a6ffd700 1 -- 172.21.15.92:0/1218706721 <== mgr.14120 v2:172.21.15.92:6800/1866676846 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f93b01033e0 con 0x7f939c038ed0 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f939c038ed0 msgr2=0x7f939c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 --2- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f939c038ed0 0x7f939c03b390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93a0006fd0 tx=0x7f93a0006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 msgr2=0x7f93b019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 --2- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b019bc10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009fb0 tx=0x7f93ac011bf0 comp rx=0 tx=0).stop 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 shutdown_connections 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 --2- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f939c038ed0 0x7f939c03b390 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 --2- 172.21.15.92:0/1218706721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b01086f0 0x7f93b019bc10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 >> 172.21.15.92:0/1218706721 conn(0x7f93b0102140 msgr2=0x7f93b0102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 shutdown_connections 2024-03-19T01:43:27.677 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:26.305+0000 7f93b8245700 1 -- 172.21.15.92:0/1218706721 wait complete. 2024-03-19T01:43:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:28 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:28 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:28.930 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.125+0000 7f831d1ce700 1 Processor -- start 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.125+0000 7f831d1ce700 1 -- start start 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.125+0000 7f831d1ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f8318106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.125+0000 7f831d1ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8318107590 con 0x7f8318106ba0 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.125+0000 7f8316d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f8318106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.125+0000 7f8316d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f8318106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57200/0 (socket says 172.21.15.92:57200) 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.125+0000 7f8316d9d700 1 -- 172.21.15.92:0/1594403244 learned_addr learned my addr 172.21.15.92:0/1594403244 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.126+0000 7f8316d9d700 1 -- 172.21.15.92:0/1594403244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8318107da0 con 0x7f8318106ba0 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.126+0000 7f8316d9d700 1 --2- 172.21.15.92:0/1594403244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f8318106fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098a0 tx=0x7f830c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a13796140fa380 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.126+0000 7f8315d9b700 1 -- 172.21.15.92:0/1594403244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f830c004160 con 0x7f8318106ba0 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.126+0000 7f8315d9b700 1 -- 172.21.15.92:0/1594403244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830c004300 con 0x7f8318106ba0 2024-03-19T01:43:28.931 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.126+0000 7f8315d9b700 1 -- 172.21.15.92:0/1594403244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f830c011b20 con 0x7f8318106ba0 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.127+0000 7f831d1ce700 1 -- 172.21.15.92:0/1594403244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 msgr2=0x7f8318106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.127+0000 7f831d1ce700 1 --2- 172.21.15.92:0/1594403244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f8318106fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098a0 tx=0x7f830c009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.127+0000 7f831d1ce700 1 -- 172.21.15.92:0/1594403244 shutdown_connections 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.127+0000 7f831d1ce700 1 --2- 172.21.15.92:0/1594403244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f8318106fc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.127+0000 7f831d1ce700 1 -- 172.21.15.92:0/1594403244 >> 172.21.15.92:0/1594403244 conn(0x7f8318102140 msgr2=0x7f8318104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.127+0000 7f831d1ce700 1 -- 172.21.15.92:0/1594403244 shutdown_connections 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.127+0000 7f831d1ce700 1 -- 172.21.15.92:0/1594403244 wait complete. 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.128+0000 7f831d1ce700 1 Processor -- start 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.128+0000 7f831d1ce700 1 -- start start 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.128+0000 7f831d1ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f831807b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.128+0000 7f831d1ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f831810b150 con 0x7f8318106ba0 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.128+0000 7f8316d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f831807b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.128+0000 7f8316d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f831807b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57202/0 (socket says 172.21.15.92:57202) 2024-03-19T01:43:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.128+0000 7f8316d9d700 1 -- 172.21.15.92:0/3488823866 learned_addr learned my addr 172.21.15.92:0/3488823866 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.128+0000 7f8316d9d700 1 -- 172.21.15.92:0/3488823866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831819e940 con 0x7f8318106ba0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.128+0000 7f8316d9d700 1 --2- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f831807b9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f830c000c00 tx=0x7f830c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.129+0000 7f82fffff700 1 -- 172.21.15.92:0/3488823866 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f830c003b60 con 0x7f8318106ba0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.129+0000 7f82fffff700 1 -- 172.21.15.92:0/3488823866 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830c01a910 con 0x7f8318106ba0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.129+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831819eb10 con 0x7f8318106ba0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.129+0000 7f82fffff700 1 -- 172.21.15.92:0/3488823866 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f830c024930 con 0x7f8318106ba0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.129+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831819eeb0 con 0x7f8318106ba0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.130+0000 7f82fffff700 1 -- 172.21.15.92:0/3488823866 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f830c022020 con 0x7f8318106ba0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.130+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8318063940 con 0x7f8318106ba0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.130+0000 7f82fffff700 1 --2- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f8300038f20 0x7f830003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.130+0000 7f82fffff700 1 -- 172.21.15.92:0/3488823866 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f830c04d080 con 0x7f8318106ba0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.130+0000 7f831659c700 1 --2- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f8300038f20 0x7f830003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.130+0000 7f831659c700 1 --2- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f8300038f20 0x7f830003b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8308006fd0 tx=0x7f8308006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.135+0000 7f82fffff700 1 -- 172.21.15.92:0/3488823866 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f830c00b3e0 con 0x7f8318106ba0 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.339+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f831819f660 con 0x7f8318106ba0 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.342+0000 7f82fffff700 1 -- 172.21.15.92:0/3488823866 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f830c0172b0 con 0x7f8318106ba0 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f8300038f20 msgr2=0x7f830003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 --2- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f8300038f20 0x7f830003b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8308006fd0 tx=0x7f8308006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 msgr2=0x7f831807b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 --2- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f831807b9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f830c000c00 tx=0x7f830c011c10 comp rx=0 tx=0).stop 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 shutdown_connections 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 --2- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f8300038f20 0x7f830003b3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 --2- 172.21.15.92:0/3488823866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8318106ba0 0x7f831807b9f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 >> 172.21.15.92:0/3488823866 conn(0x7f8318102140 msgr2=0x7f8318102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 shutdown_connections 2024-03-19T01:43:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:28.345+0000 7f831d1ce700 1 -- 172.21.15.92:0/3488823866 wait complete. 2024-03-19T01:43:29.547 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:29 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3488823866' entity='client.admin' 2024-03-19T01:43:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:30 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/207340779' entity='client.admin' 2024-03-19T01:43:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:30 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:30.841 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.327+0000 7fd5b79ca700 1 Processor -- start 2024-03-19T01:43:30.841 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.327+0000 7fd5b79ca700 1 -- start start 2024-03-19T01:43:30.841 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.327+0000 7fd5b79ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b0108070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:30.841 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.327+0000 7fd5b79ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0108640 con 0x7fd5b0107c50 2024-03-19T01:43:30.841 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.327+0000 7fd5b5766700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b0108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:30.841 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.327+0000 7fd5b5766700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b0108070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57216/0 (socket says 172.21.15.92:57216) 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.327+0000 7fd5b5766700 1 -- 172.21.15.92:0/147368398 learned_addr learned my addr 172.21.15.92:0/147368398 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5b5766700 1 -- 172.21.15.92:0/147368398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b0108e50 con 0x7fd5b0107c50 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5b5766700 1 --2- 172.21.15.92:0/147368398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b0108070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0098a0 tx=0x7fd5ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a6d9b6cfa6595e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5a7fff700 1 -- 172.21.15.92:0/147368398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5ac004160 con 0x7fd5b0107c50 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5a7fff700 1 -- 172.21.15.92:0/147368398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac004300 con 0x7fd5b0107c50 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5a7fff700 1 -- 172.21.15.92:0/147368398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5ac011b20 con 0x7fd5b0107c50 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5b79ca700 1 -- 172.21.15.92:0/147368398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 msgr2=0x7fd5b0108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5b79ca700 1 --2- 172.21.15.92:0/147368398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b0108070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0098a0 tx=0x7fd5ac009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5b79ca700 1 -- 172.21.15.92:0/147368398 shutdown_connections 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5b79ca700 1 --2- 172.21.15.92:0/147368398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b0108070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5b79ca700 1 -- 172.21.15.92:0/147368398 >> 172.21.15.92:0/147368398 conn(0x7fd5b0103210 msgr2=0x7fd5b0105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5b79ca700 1 -- 172.21.15.92:0/147368398 shutdown_connections 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.328+0000 7fd5b79ca700 1 -- 172.21.15.92:0/147368398 wait complete. 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.329+0000 7fd5b79ca700 1 Processor -- start 2024-03-19T01:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.329+0000 7fd5b79ca700 1 -- start start 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.329+0000 7fd5b79ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b007daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.329+0000 7fd5b79ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b010c200 con 0x7fd5b0107c50 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.329+0000 7fd5b5766700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b007daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.329+0000 7fd5b5766700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b007daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57222/0 (socket says 172.21.15.92:57222) 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.329+0000 7fd5b5766700 1 -- 172.21.15.92:0/207340779 learned_addr learned my addr 172.21.15.92:0/207340779 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.329+0000 7fd5b5766700 1 -- 172.21.15.92:0/207340779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b007e030 con 0x7fd5b0107c50 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.330+0000 7fd5b5766700 1 --2- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b007daf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0116a0 tx=0x7fd5ac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.330+0000 7fd5a67fc700 1 -- 172.21.15.92:0/207340779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5ac003a40 con 0x7fd5b0107c50 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.330+0000 7fd5a67fc700 1 -- 172.21.15.92:0/207340779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac003be0 con 0x7fd5b0107c50 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.330+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b007c220 con 0x7fd5b0107c50 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.330+0000 7fd5a67fc700 1 -- 172.21.15.92:0/207340779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5ac01a720 con 0x7fd5b0107c50 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.330+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b007c6e0 con 0x7fd5b0107c50 2024-03-19T01:43:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.331+0000 7fd5a67fc700 1 -- 172.21.15.92:0/207340779 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fd5ac01a9d0 con 0x7fd5b0107c50 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.331+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b0063940 con 0x7fd5b0107c50 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.331+0000 7fd5a67fc700 1 --2- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fd59c038f20 0x7fd59c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.331+0000 7fd5a67fc700 1 -- 172.21.15.92:0/207340779 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd5ac04d610 con 0x7fd5b0107c50 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.331+0000 7fd5b4f65700 1 --2- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fd59c038f20 0x7fd59c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.332+0000 7fd5b4f65700 1 --2- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fd59c038f20 0x7fd59c03b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0006fd0 tx=0x7fd5a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.336+0000 7fd5a67fc700 1 -- 172.21.15.92:0/207340779 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd5ac011e20 con 0x7fd5b0107c50 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.547+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fd5b007ca20 con 0x7fd5b0107c50 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.550+0000 7fd5a67fc700 1 -- 172.21.15.92:0/207340779 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fd5b007ca20 con 0x7fd5b0107c50 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fd59c038f20 msgr2=0x7fd59c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 --2- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fd59c038f20 0x7fd59c03b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0006fd0 tx=0x7fd5a0006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 msgr2=0x7fd5b007daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 --2- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b007daf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0116a0 tx=0x7fd5ac011ba0 comp rx=0 tx=0).stop 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 shutdown_connections 2024-03-19T01:43:30.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 --2- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fd59c038f20 0x7fd59c03b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:30.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 --2- 172.21.15.92:0/207340779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5b0107c50 0x7fd5b007daf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:30.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 >> 172.21.15.92:0/207340779 conn(0x7fd5b0103210 msgr2=0x7fd5b0103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:30.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 shutdown_connections 2024-03-19T01:43:30.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:29.552+0000 7fd5b79ca700 1 -- 172.21.15.92:0/207340779 wait complete. 2024-03-19T01:43:30.845 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling the dashboard module... 2024-03-19T01:43:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:32 smithi092 ceph-mon[62574]: from='mgr.14120 172.21.15.92:0/815355771' entity='mgr.x' 2024-03-19T01:43:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:32 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3784939765' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-19T01:43:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:32 smithi092 ceph-mon[62574]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:43:32.811 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:32 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: ignoring --setuser ceph since I am not root 2024-03-19T01:43:32.811 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:32 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: ignoring --setgroup ceph since I am not root 2024-03-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.281+0000 7fa0a61b5700 1 Processor -- start 2024-03-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.281+0000 7fa0a61b5700 1 -- start start 2024-03-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.281+0000 7fa0a61b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a0108090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.281+0000 7fa0a61b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0108660 con 0x7fa0a0107c70 2024-03-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.281+0000 7fa09f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a0108090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:33.110 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.281+0000 7fa09f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a0108090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57238/0 (socket says 172.21.15.92:57238) 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.281+0000 7fa09f7fe700 1 -- 172.21.15.92:0/718609602 learned_addr learned my addr 172.21.15.92:0/718609602 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.282+0000 7fa09f7fe700 1 -- 172.21.15.92:0/718609602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a0108e70 con 0x7fa0a0107c70 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.282+0000 7fa09f7fe700 1 --2- 172.21.15.92:0/718609602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a0108090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0940098a0 tx=0x7fa094009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790c651fd5565fa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.282+0000 7fa09e7fc700 1 -- 172.21.15.92:0/718609602 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa094004160 con 0x7fa0a0107c70 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.282+0000 7fa09e7fc700 1 -- 172.21.15.92:0/718609602 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa094004300 con 0x7fa0a0107c70 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.282+0000 7fa09e7fc700 1 -- 172.21.15.92:0/718609602 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa094011b20 con 0x7fa0a0107c70 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.283+0000 7fa0a61b5700 1 -- 172.21.15.92:0/718609602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 msgr2=0x7fa0a0108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.283+0000 7fa0a61b5700 1 --2- 172.21.15.92:0/718609602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a0108090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0940098a0 tx=0x7fa094009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.283+0000 7fa0a61b5700 1 -- 172.21.15.92:0/718609602 shutdown_connections 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.283+0000 7fa0a61b5700 1 --2- 172.21.15.92:0/718609602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a0108090 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.283+0000 7fa0a61b5700 1 -- 172.21.15.92:0/718609602 >> 172.21.15.92:0/718609602 conn(0x7fa0a0103210 msgr2=0x7fa0a0105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.283+0000 7fa0a61b5700 1 -- 172.21.15.92:0/718609602 shutdown_connections 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.283+0000 7fa0a61b5700 1 -- 172.21.15.92:0/718609602 wait complete. 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.283+0000 7fa0a61b5700 1 Processor -- start 2024-03-19T01:43:33.111 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa0a61b5700 1 -- start start 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa0a61b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a007daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa0a61b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a010c220 con 0x7fa0a0107c70 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa09f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a007daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa09f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a007daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57246/0 (socket says 172.21.15.92:57246) 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa09f7fe700 1 -- 172.21.15.92:0/3784939765 learned_addr learned my addr 172.21.15.92:0/3784939765 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa09f7fe700 1 -- 172.21.15.92:0/3784939765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a007dfe0 con 0x7fa0a0107c70 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa09f7fe700 1 --2- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a007daa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0940116a0 tx=0x7fa094011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa09cff9700 1 -- 172.21.15.92:0/3784939765 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa094003a40 con 0x7fa0a0107c70 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.284+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a007c220 con 0x7fa0a0107c70 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.285+0000 7fa09cff9700 1 -- 172.21.15.92:0/3784939765 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa094003be0 con 0x7fa0a0107c70 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.285+0000 7fa09cff9700 1 -- 172.21.15.92:0/3784939765 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa09401a720 con 0x7fa0a0107c70 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.285+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a007c680 con 0x7fa0a0107c70 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.285+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a0063940 con 0x7fa0a0107c70 2024-03-19T01:43:33.112 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.286+0000 7fa09cff9700 1 -- 172.21.15.92:0/3784939765 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fa094024aa0 con 0x7fa0a0107c70 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.286+0000 7fa09cff9700 1 --2- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa088038ed0 0x7fa08803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.286+0000 7fa09cff9700 1 -- 172.21.15.92:0/3784939765 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa09404d7e0 con 0x7fa0a0107c70 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.287+0000 7fa09effd700 1 --2- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa088038ed0 0x7fa08803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.287+0000 7fa09effd700 1 --2- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa088038ed0 0x7fa08803b390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa090006fd0 tx=0x7fa090006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.291+0000 7fa09cff9700 1 -- 172.21.15.92:0/3784939765 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa094011e20 con 0x7fa0a0107c70 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:31.555+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa0a007c900 con 0x7fa0a0107c70 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.515+0000 7fa09cff9700 1 -- 172.21.15.92:0/3784939765 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa094016260 con 0x7fa0a0107c70 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.516+0000 7fa09cff9700 1 -- 172.21.15.92:0/3784939765 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7fa09404c1d0 con 0x7fa0a0107c70 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.517+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa088038ed0 msgr2=0x7fa08803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.517+0000 7fa0a61b5700 1 --2- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa088038ed0 0x7fa08803b390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa090006fd0 tx=0x7fa090006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.517+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 msgr2=0x7fa0a007daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.517+0000 7fa0a61b5700 1 --2- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a007daa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0940116a0 tx=0x7fa094011ba0 comp rx=0 tx=0).stop 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.518+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 shutdown_connections 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.518+0000 7fa0a61b5700 1 --2- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7fa088038ed0 0x7fa08803b390 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:33.113 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.518+0000 7fa0a61b5700 1 --2- 172.21.15.92:0/3784939765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0a0107c70 0x7fa0a007daa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.518+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 >> 172.21.15.92:0/3784939765 conn(0x7fa0a0103210 msgr2=0x7fa0a0103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.518+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 shutdown_connections 2024-03-19T01:43:33.114 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:32.518+0000 7fa0a61b5700 1 -- 172.21.15.92:0/3784939765 wait complete. 2024-03-19T01:43:33.513 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:33 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:33.082+0000 7fcf9d480200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:43:33.513 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:33 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:33.183+0000 7fcf9d480200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:43:33.767 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:33 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3784939765' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-19T01:43:33.767 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:33 smithi092 ceph-mon[62574]: mgrmap e9: x(active, since 20s) 2024-03-19T01:43:34.060 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:33 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:33.826+0000 7fcf9d480200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:43:34.060 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:33 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:33.921+0000 7fcf9d480200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:43:34.519 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:34.082+0000 7fcf9d480200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:43:34.520 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:34.178+0000 7fcf9d480200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:43:34.520 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:34.263+0000 7fcf9d480200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:43:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:34 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/448385701' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-19T01:43:34.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:34.532+0000 7fcf9d480200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:43:34.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:34.644+0000 7fcf9d480200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:43:34.811 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:34.730+0000 7fcf9d480200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:43:34.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-03-19T01:43:34.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-19T01:43:34.844 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.964+0000 7f0ef69b3700 1 Processor -- start 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.964+0000 7f0ef69b3700 1 -- start start 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.964+0000 7f0ef69b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef010ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.964+0000 7f0ef69b3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef007bdf0 con 0x7f0ef0108750 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.965+0000 7f0eeffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef010ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.965+0000 7f0eeffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef010ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51076/0 (socket says 172.21.15.92:51076) 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.965+0000 7f0eeffff700 1 -- 172.21.15.92:0/578109902 learned_addr learned my addr 172.21.15.92:0/578109902 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.965+0000 7f0eeffff700 1 -- 172.21.15.92:0/578109902 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef010b080 con 0x7f0ef0108750 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.965+0000 7f0eeffff700 1 --2- 172.21.15.92:0/578109902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef010ab40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009ab0 tx=0x7f0ed8009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62314403980e272 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.965+0000 7f0eeeffd700 1 -- 172.21.15.92:0/578109902 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed8004160 con 0x7f0ef0108750 2024-03-19T01:43:34.845 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.965+0000 7f0eeeffd700 1 -- 172.21.15.92:0/578109902 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed800bcc0 con 0x7f0ef0108750 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.965+0000 7f0eeeffd700 1 -- 172.21.15.92:0/578109902 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed8003d00 con 0x7f0ef0108750 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.966+0000 7f0ef69b3700 1 -- 172.21.15.92:0/578109902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 msgr2=0x7f0ef010ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.966+0000 7f0ef69b3700 1 --2- 172.21.15.92:0/578109902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef010ab40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009ab0 tx=0x7f0ed8009dc0 comp rx=0 tx=0).stop 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.966+0000 7f0ef69b3700 1 -- 172.21.15.92:0/578109902 shutdown_connections 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.966+0000 7f0ef69b3700 1 --2- 172.21.15.92:0/578109902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef010ab40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.966+0000 7f0ef69b3700 1 -- 172.21.15.92:0/578109902 >> 172.21.15.92:0/578109902 conn(0x7f0ef0102140 msgr2=0x7f0ef0104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.966+0000 7f0ef69b3700 1 -- 172.21.15.92:0/578109902 shutdown_connections 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.966+0000 7f0ef69b3700 1 -- 172.21.15.92:0/578109902 wait complete. 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.967+0000 7f0ef69b3700 1 Processor -- start 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.967+0000 7f0ef69b3700 1 -- start start 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.967+0000 7f0ef69b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.967+0000 7f0ef69b3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010c0e0 con 0x7f0ef0108750 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.967+0000 7f0eeffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.967+0000 7f0eeffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51078/0 (socket says 172.21.15.92:51078) 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.967+0000 7f0eeffff700 1 -- 172.21.15.92:0/448385701 learned_addr learned my addr 172.21.15.92:0/448385701 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.967+0000 7f0eeffff700 1 -- 172.21.15.92:0/448385701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef019c190 con 0x7f0ef0108750 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.968+0000 7f0eeffff700 1 --2- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef019bc50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8003a40 tx=0x7f0ed800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.968+0000 7f0eed7fa700 1 -- 172.21.15.92:0/448385701 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed8004300 con 0x7f0ef0108750 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.968+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef019c420 con 0x7f0ef0108750 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.968+0000 7f0eed7fa700 1 -- 172.21.15.92:0/448385701 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed80044a0 con 0x7f0ef0108750 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.968+0000 7f0eed7fa700 1 -- 172.21.15.92:0/448385701 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ed8011740 con 0x7f0ef0108750 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.968+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef019f110 con 0x7f0ef0108750 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.969+0000 7f0eed7fa700 1 -- 172.21.15.92:0/448385701 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f0ed8022020 con 0x7f0ef0108750 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.969+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef0063940 con 0x7f0ef0108750 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.969+0000 7f0eed7fa700 1 --2- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f0edc038f20 0x7f0edc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.969+0000 7f0eed7fa700 1 -- 172.21.15.92:0/448385701 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0ed804d0b0 con 0x7f0ef0108750 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.969+0000 7f0eef7fe700 1 -- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f0edc038f20 msgr2=0x7f0edc03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1866676846 2024-03-19T01:43:34.847 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.969+0000 7f0eef7fe700 1 --2- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f0edc038f20 0x7f0edc03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:33.974+0000 7f0eed7fa700 1 -- 172.21.15.92:0/448385701 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0ed80199c0 con 0x7f0ef0108750 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.170+0000 7f0eef7fe700 1 -- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f0edc038f20 msgr2=0x7f0edc03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1866676846 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.170+0000 7f0eef7fe700 1 --2- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f0edc038f20 0x7f0edc03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.250+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f0ef019f6c0 con 0x7f0ef0108750 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.250+0000 7f0eed7fa700 1 -- 172.21.15.92:0/448385701 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f0ed8019c20 con 0x7f0ef0108750 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f0edc038f20 msgr2=0x7f0edc03b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 --2- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f0edc038f20 0x7f0edc03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 msgr2=0x7f0ef019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 --2- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef019bc50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8003a40 tx=0x7f0ed800bac0 comp rx=0 tx=0).stop 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 shutdown_connections 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 --2- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f0edc038f20 0x7f0edc03b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 --2- 172.21.15.92:0/448385701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ef0108750 0x7f0ef019bc50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 >> 172.21.15.92:0/448385701 conn(0x7f0ef0102140 msgr2=0x7f0ef0102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 shutdown_connections 2024-03-19T01:43:34.848 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:34.253+0000 7f0ef69b3700 1 -- 172.21.15.92:0/448385701 wait complete. 2024-03-19T01:43:34.849 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for the mgr to restart... 2024-03-19T01:43:34.849 INFO:teuthology.orchestra.run.smithi092.stdout:Waiting for mgr epoch 9... 2024-03-19T01:43:35.164 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:34.827+0000 7fcf9d480200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:43:35.164 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:35 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:35.009+0000 7fcf9d480200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:43:36.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:35 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:35.974+0000 7fcf9d480200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:43:36.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:36.103+0000 7fcf9d480200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:43:36.719 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:36.430+0000 7fcf9d480200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:43:36.720 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:36.518+0000 7fcf9d480200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:43:36.720 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:36.719+0000 7fcf9d480200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:43:37.060 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:36.902+0000 7fcf9d480200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:43:37.446 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:37 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:37.192+0000 7fcf9d480200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:43:37.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:37 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:37.445+0000 7fcf9d480200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:43:37.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:37 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:37.680+0000 7fcf9d480200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:43:38.560 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:38 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:38.476+0000 7fcf9d480200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:43:39.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:39.036+0000 7fcf9d480200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:43:39.310 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:39.134+0000 7fcf9d480200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:43:39.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:43:39.349+0000 7fcf9d480200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:43:39.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: Active manager daemon x restarted 2024-03-19T01:43:39.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: Activating manager daemon x 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: osdmap e3: 0 total, 0 up, 0 in 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: mgrmap e10: x(active, starting, since 0.00349662s) 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: Manager daemon x is now available 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:40.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-19T01:43:40.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-19T01:43:40.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:40 smithi092 ceph-mon[62574]: mgrmap e11: x(active, since 1.00636s) 2024-03-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout { 2024-03-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout } 2024-03-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.265+0000 7f2696e7b700 1 Processor -- start 2024-03-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.265+0000 7f2696e7b700 1 -- start start 2024-03-19T01:43:41.338 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.265+0000 7f2696e7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f269010ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.265+0000 7f2696e7b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f269007be10 con 0x7f2690108730 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.265+0000 7f2694c17700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f269010ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.265+0000 7f2694c17700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f269010ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51090/0 (socket says 172.21.15.92:51090) 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.265+0000 7f2694c17700 1 -- 172.21.15.92:0/1785775829 learned_addr learned my addr 172.21.15.92:0/1785775829 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.265+0000 7f2694c17700 1 -- 172.21.15.92:0/1785775829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269010b060 con 0x7f2690108730 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.265+0000 7f2694c17700 1 --2- 172.21.15.92:0/1785775829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f269010ab20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26840098a0 tx=0x7f2684009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877086873b604e7e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f268f7fe700 1 -- 172.21.15.92:0/1785775829 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2684004160 con 0x7f2690108730 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f268f7fe700 1 -- 172.21.15.92:0/1785775829 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2684004300 con 0x7f2690108730 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f268f7fe700 1 -- 172.21.15.92:0/1785775829 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2684011b20 con 0x7f2690108730 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f2696e7b700 1 -- 172.21.15.92:0/1785775829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 msgr2=0x7f269010ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f2696e7b700 1 --2- 172.21.15.92:0/1785775829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f269010ab20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26840098a0 tx=0x7f2684009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f2696e7b700 1 -- 172.21.15.92:0/1785775829 shutdown_connections 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f2696e7b700 1 --2- 172.21.15.92:0/1785775829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f269010ab20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f2696e7b700 1 -- 172.21.15.92:0/1785775829 >> 172.21.15.92:0/1785775829 conn(0x7f2690102140 msgr2=0x7f2690104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:41.339 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f2696e7b700 1 -- 172.21.15.92:0/1785775829 shutdown_connections 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.266+0000 7f2696e7b700 1 -- 172.21.15.92:0/1785775829 wait complete. 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.267+0000 7f2696e7b700 1 Processor -- start 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.267+0000 7f2696e7b700 1 -- start start 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.267+0000 7f2696e7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f2690107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.267+0000 7f2696e7b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f269010c0c0 con 0x7f2690108730 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.267+0000 7f2694c17700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f2690107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.267+0000 7f2694c17700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f2690107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51106/0 (socket says 172.21.15.92:51106) 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.267+0000 7f2694c17700 1 -- 172.21.15.92:0/131368135 learned_addr learned my addr 172.21.15.92:0/131368135 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.268+0000 7f2694c17700 1 -- 172.21.15.92:0/131368135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26901084f0 con 0x7f2690108730 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.268+0000 7f2694c17700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f2690107fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2684011b60 tx=0x7f2684011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.268+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2684003b60 con 0x7f2690108730 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.268+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268401a910 con 0x7f2690108730 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.268+0000 7f2696e7b700 1 -- 172.21.15.92:0/131368135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26901066e0 con 0x7f2690108730 2024-03-19T01:43:41.340 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.268+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2684024930 con 0x7f2690108730 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.268+0000 7f2696e7b700 1 -- 172.21.15.92:0/131368135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2690106ba0 con 0x7f2690108730 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.269+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f2684022020 con 0x7f2690108730 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.269+0000 7f268dffb700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 0x7f268003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.269+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 --> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f268003bba0 con 0x7f2680038fd0 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.269+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f268404d060 con 0x7f2690108730 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.269+0000 7f268ffff700 1 -- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 msgr2=0x7f268003b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1866676846 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.269+0000 7f268ffff700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 0x7f268003b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.470+0000 7f268ffff700 1 -- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 msgr2=0x7f268003b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1866676846 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.470+0000 7f268ffff700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 0x7f268003b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.871+0000 7f268ffff700 1 -- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 msgr2=0x7f268003b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1866676846 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:35.871+0000 7f268ffff700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 0x7f268003b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:36.672+0000 7f268ffff700 1 -- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 msgr2=0x7f268003b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1866676846 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:36.672+0000 7f268ffff700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 0x7f268003b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:38.274+0000 7f268ffff700 1 -- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 msgr2=0x7f268003b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.92:6800/1866676846 2024-03-19T01:43:41.341 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:38.274+0000 7f268ffff700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 0x7f268003b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:39.365+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mgrmap(e 10) ==== 45886+0+0 (secure 0 0 0) 0x7f2684024ad0 con 0x7f2690108730 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:39.365+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 msgr2=0x7f268003b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:39.365+0000 7f268dffb700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 0x7f268003b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.366+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7f268404f070 con 0x7f2690108730 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.366+0000 7f268dffb700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f268003ca80 0x7f268003ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.366+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2684001d30 con 0x7f268003ca80 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.367+0000 7f268ffff700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f268003ca80 0x7f268003ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.367+0000 7f268ffff700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f268003ca80 0x7f268003ee70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f267c003a10 tx=0x7f267c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.368+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f2684001d30 con 0x7f268003ca80 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.380+0000 7f2696e7b700 1 -- 172.21.15.92:0/131368135 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f269010c240 con 0x7f268003ca80 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.380+0000 7f268dffb700 1 -- 172.21.15.92:0/131368135 <== mgr.14152 v2:172.21.15.92:6800/3230736370 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f269010c240 con 0x7f268003ca80 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 -- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f268003ca80 msgr2=0x7f268003ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f268003ca80 0x7f268003ee70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f267c003a10 tx=0x7f267c009270 comp rx=0 tx=0).stop 2024-03-19T01:43:41.342 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 -- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 msgr2=0x7f2690107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f2690107fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2684011b60 tx=0x7f2684011c40 comp rx=0 tx=0).stop 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 -- 172.21.15.92:0/131368135 shutdown_connections 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f268003ca80 0x7f268003ee70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2690108730 0x7f2690107fb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 --2- 172.21.15.92:0/131368135 >> [v2:172.21.15.92:6800/1866676846,v1:172.21.15.92:6801/1866676846] conn(0x7f2680038fd0 0x7f268003b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 -- 172.21.15.92:0/131368135 >> 172.21.15.92:0/131368135 conn(0x7f2690102140 msgr2=0x7f2690102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 -- 172.21.15.92:0/131368135 shutdown_connections 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:40.381+0000 7f2696e7b700 1 -- 172.21.15.92:0/131368135 wait complete. 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:mgr epoch 9 is available 2024-03-19T01:43:41.343 INFO:teuthology.orchestra.run.smithi092.stdout:Generating a dashboard self-signed certificate... 2024-03-19T01:43:41.582 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:41 smithi092 ceph-mon[62574]: [19/Mar/2024:01:43:39] ENGINE Bus STARTING 2024-03-19T01:43:41.583 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:41 smithi092 ceph-mon[62574]: [19/Mar/2024:01:43:39] ENGINE Serving on http://172.21.15.92:8765 2024-03-19T01:43:41.583 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:41 smithi092 ceph-mon[62574]: [19/Mar/2024:01:43:40] ENGINE Serving on https://172.21.15.92:7150 2024-03-19T01:43:41.583 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:41 smithi092 ceph-mon[62574]: [19/Mar/2024:01:43:40] ENGINE Bus STARTED 2024-03-19T01:43:41.583 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:41 smithi092 ceph-mon[62574]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-19T01:43:41.583 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:41 smithi092 ceph-mon[62574]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-19T01:43:42.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:42 smithi092 ceph-mon[62574]: mgrmap e12: x(active, since 2s) 2024-03-19T01:43:42.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:42.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:42.678 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:42.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-19T01:43:42.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.794+0000 7f29ea87e700 1 Processor -- start 2024-03-19T01:43:42.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.794+0000 7f29ea87e700 1 -- start start 2024-03-19T01:43:42.686 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.794+0000 7f29ea87e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e4108070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.794+0000 7f29ea87e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4108640 con 0x7f29e4107c50 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.794+0000 7f29e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e4108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.794+0000 7f29e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e4108070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51174/0 (socket says 172.21.15.92:51174) 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.794+0000 7f29e3fff700 1 -- 172.21.15.92:0/218436175 learned_addr learned my addr 172.21.15.92:0/218436175 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.795+0000 7f29e3fff700 1 -- 172.21.15.92:0/218436175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e4108e50 con 0x7f29e4107c50 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.795+0000 7f29e3fff700 1 --2- 172.21.15.92:0/218436175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e4108070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29d40098a0 tx=0x7f29d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac61add65033c184 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.795+0000 7f29e2ffd700 1 -- 172.21.15.92:0/218436175 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d4004160 con 0x7f29e4107c50 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.795+0000 7f29e2ffd700 1 -- 172.21.15.92:0/218436175 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d4004300 con 0x7f29e4107c50 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.795+0000 7f29e2ffd700 1 -- 172.21.15.92:0/218436175 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d4011b20 con 0x7f29e4107c50 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.796+0000 7f29ea87e700 1 -- 172.21.15.92:0/218436175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 msgr2=0x7f29e4108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.796+0000 7f29ea87e700 1 --2- 172.21.15.92:0/218436175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e4108070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29d40098a0 tx=0x7f29d4009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.796+0000 7f29ea87e700 1 -- 172.21.15.92:0/218436175 shutdown_connections 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.796+0000 7f29ea87e700 1 --2- 172.21.15.92:0/218436175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e4108070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.796+0000 7f29ea87e700 1 -- 172.21.15.92:0/218436175 >> 172.21.15.92:0/218436175 conn(0x7f29e4103210 msgr2=0x7f29e4105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.796+0000 7f29ea87e700 1 -- 172.21.15.92:0/218436175 shutdown_connections 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.796+0000 7f29ea87e700 1 -- 172.21.15.92:0/218436175 wait complete. 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.796+0000 7f29ea87e700 1 Processor -- start 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.796+0000 7f29ea87e700 1 -- start start 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29ea87e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29ea87e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f29e410c200 con 0x7f29e4107c50 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51180/0 (socket says 172.21.15.92:51180) 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29e3fff700 1 -- 172.21.15.92:0/3461526860 learned_addr learned my addr 172.21.15.92:0/3461526860 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29e3fff700 1 -- 172.21.15.92:0/3461526860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e419c120 con 0x7f29e4107c50 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29e3fff700 1 --2- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e419bbe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29d40116a0 tx=0x7f29d4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29e17fa700 1 -- 172.21.15.92:0/3461526860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d4003a40 con 0x7f29e4107c50 2024-03-19T01:43:42.688 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e419c3b0 con 0x7f29e4107c50 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29e17fa700 1 -- 172.21.15.92:0/3461526860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d4003be0 con 0x7f29e4107c50 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29e17fa700 1 -- 172.21.15.92:0/3461526860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d401a720 con 0x7f29e4107c50 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.797+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e419c810 con 0x7f29e4107c50 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.798+0000 7f29e17fa700 1 -- 172.21.15.92:0/3461526860 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f29d401aa20 con 0x7f29e4107c50 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.798+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e4063940 con 0x7f29e4107c50 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.798+0000 7f29e17fa700 1 --2- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29cc038f20 0x7f29cc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.798+0000 7f29e17fa700 1 -- 172.21.15.92:0/3461526860 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f29d404d740 con 0x7f29e4107c50 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.799+0000 7f29e37fe700 1 --2- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29cc038f20 0x7f29cc03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.799+0000 7f29e37fe700 1 --2- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29cc038f20 0x7f29cc03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29d8006fd0 tx=0x7f29d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:41.803+0000 7f29e17fa700 1 -- 172.21.15.92:0/3461526860 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f29d4011e20 con 0x7f29e4107c50 2024-03-19T01:43:42.689 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.013+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f29e41044a0 con 0x7f29cc038f20 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.071+0000 7f29e17fa700 1 -- 172.21.15.92:0/3461526860 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f29e41044a0 con 0x7f29cc038f20 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29cc038f20 msgr2=0x7f29cc03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 --2- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29cc038f20 0x7f29cc03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29d8006fd0 tx=0x7f29d8006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 msgr2=0x7f29e419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 --2- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e419bbe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29d40116a0 tx=0x7f29d4011ba0 comp rx=0 tx=0).stop 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 shutdown_connections 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 --2- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29cc038f20 0x7f29cc03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 --2- 172.21.15.92:0/3461526860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29e4107c50 0x7f29e419bbe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 >> 172.21.15.92:0/3461526860 conn(0x7f29e4103210 msgr2=0x7f29e4103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 shutdown_connections 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:42.075+0000 7f29ea87e700 1 -- 172.21.15.92:0/3461526860 wait complete. 2024-03-19T01:43:42.690 INFO:teuthology.orchestra.run.smithi092.stdout:Creating initial admin user... 2024-03-19T01:43:43.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:43 smithi092 ceph-mon[62574]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$MOMSaOm9qNFn5VFB6wTDF.u7C1J6eWkUU.T08SoykXOg4o5XU2i.O", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710812623, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-19T01:43:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.111+0000 7fc198635700 1 Processor -- start 2024-03-19T01:43:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.111+0000 7fc198635700 1 -- start start 2024-03-19T01:43:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.111+0000 7fc198635700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc190106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.111+0000 7fc198635700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc190107570 con 0x7fc190106b80 2024-03-19T01:43:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.112+0000 7fc1963d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc190106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.112+0000 7fc1963d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc190106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51184/0 (socket says 172.21.15.92:51184) 2024-03-19T01:43:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.112+0000 7fc1963d1700 1 -- 172.21.15.92:0/446931832 learned_addr learned my addr 172.21.15.92:0/446931832 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:44.201 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.112+0000 7fc1963d1700 1 -- 172.21.15.92:0/446931832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc190107d80 con 0x7fc190106b80 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.112+0000 7fc1963d1700 1 --2- 172.21.15.92:0/446931832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc190106fa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc180009e80 tx=0x7fc18000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8242a6f3c2b52e0b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.112+0000 7fc1953cf700 1 -- 172.21.15.92:0/446931832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc180017070 con 0x7fc190106b80 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.112+0000 7fc1953cf700 1 -- 172.21.15.92:0/446931832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18000b8c0 con 0x7fc190106b80 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.112+0000 7fc1953cf700 1 -- 172.21.15.92:0/446931832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1800045c0 con 0x7fc190106b80 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.113+0000 7fc198635700 1 -- 172.21.15.92:0/446931832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 msgr2=0x7fc190106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.113+0000 7fc198635700 1 --2- 172.21.15.92:0/446931832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc190106fa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc180009e80 tx=0x7fc18000b330 comp rx=0 tx=0).stop 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.113+0000 7fc198635700 1 -- 172.21.15.92:0/446931832 shutdown_connections 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.113+0000 7fc198635700 1 --2- 172.21.15.92:0/446931832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc190106fa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.113+0000 7fc198635700 1 -- 172.21.15.92:0/446931832 >> 172.21.15.92:0/446931832 conn(0x7fc190102140 msgr2=0x7fc190104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.113+0000 7fc198635700 1 -- 172.21.15.92:0/446931832 shutdown_connections 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.113+0000 7fc198635700 1 -- 172.21.15.92:0/446931832 wait complete. 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc198635700 1 Processor -- start 2024-03-19T01:43:44.202 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc198635700 1 -- start start 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc198635700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc19019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc198635700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc19010b130 con 0x7fc190106b80 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc1963d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc19019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc1963d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc19019bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51196/0 (socket says 172.21.15.92:51196) 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc1963d1700 1 -- 172.21.15.92:0/2495314036 learned_addr learned my addr 172.21.15.92:0/2495314036 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc1963d1700 1 -- 172.21.15.92:0/2495314036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19019c170 con 0x7fc190106b80 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc1963d1700 1 --2- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc19019bc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc180003fa0 tx=0x7fc180004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc18f7fe700 1 -- 172.21.15.92:0/2495314036 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc180020020 con 0x7fc190106b80 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc18f7fe700 1 -- 172.21.15.92:0/2495314036 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18000bba0 con 0x7fc190106b80 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19019c400 con 0x7fc190106b80 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.114+0000 7fc18f7fe700 1 -- 172.21.15.92:0/2495314036 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc180017040 con 0x7fc190106b80 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.115+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19019f0f0 con 0x7fc190106b80 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.115+0000 7fc18f7fe700 1 -- 172.21.15.92:0/2495314036 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7fc180023020 con 0x7fc190106b80 2024-03-19T01:43:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.115+0000 7fc18f7fe700 1 --2- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc178038f20 0x7fc17803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.115+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc190063940 con 0x7fc190106b80 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.115+0000 7fc18f7fe700 1 -- 172.21.15.92:0/2495314036 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc18000f050 con 0x7fc190106b80 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.116+0000 7fc195bd0700 1 --2- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc178038f20 0x7fc17803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.116+0000 7fc195bd0700 1 --2- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc178038f20 0x7fc17803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc184006fd0 tx=0x7fc184006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.120+0000 7fc18f7fe700 1 -- 172.21.15.92:0/2495314036 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc18002a020 con 0x7fc190106b80 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.349+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc190195c20 con 0x7fc178038f20 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.577+0000 7fc18f7fe700 1 -- 172.21.15.92:0/2495314036 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc190195c20 con 0x7fc178038f20 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc178038f20 msgr2=0x7fc17803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 --2- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc178038f20 0x7fc17803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc184006fd0 tx=0x7fc184006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 msgr2=0x7fc19019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 --2- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc19019bc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc180003fa0 tx=0x7fc180004080 comp rx=0 tx=0).stop 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 shutdown_connections 2024-03-19T01:43:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 --2- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc178038f20 0x7fc17803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:44.205 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 --2- 172.21.15.92:0/2495314036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc190106b80 0x7fc19019bc30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:44.205 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 >> 172.21.15.92:0/2495314036 conn(0x7fc190102140 msgr2=0x7fc190103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:44.205 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 shutdown_connections 2024-03-19T01:43:44.205 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:43.580+0000 7fc198635700 1 -- 172.21.15.92:0/2495314036 wait complete. 2024-03-19T01:43:44.205 INFO:teuthology.orchestra.run.smithi092.stdout:Fetching dashboard port number... 2024-03-19T01:43:44.786 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:44 smithi092 ceph-mon[62574]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:44.786 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:44.786 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:45.550 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stdout 8443 2024-03-19T01:43:45.550 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.621+0000 7ff89c482700 1 Processor -- start 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.621+0000 7ff89c482700 1 -- start start 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.621+0000 7ff89c482700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff89410ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.621+0000 7ff89c482700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff89407bdf0 con 0x7ff894108780 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.621+0000 7ff89a21e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff89410ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.622+0000 7ff89a21e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff89410ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44792/0 (socket says 172.21.15.92:44792) 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.622+0000 7ff89a21e700 1 -- 172.21.15.92:0/2172755425 learned_addr learned my addr 172.21.15.92:0/2172755425 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.622+0000 7ff89a21e700 1 -- 172.21.15.92:0/2172755425 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89410b0b0 con 0x7ff894108780 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.622+0000 7ff89a21e700 1 --2- 172.21.15.92:0/2172755425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff89410ab70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8840098a0 tx=0x7ff884009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4118e6768f228b20 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.622+0000 7ff89921c700 1 -- 172.21.15.92:0/2172755425 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff884004160 con 0x7ff894108780 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.622+0000 7ff89921c700 1 -- 172.21.15.92:0/2172755425 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff884004300 con 0x7ff894108780 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.622+0000 7ff89921c700 1 -- 172.21.15.92:0/2172755425 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff884011b20 con 0x7ff894108780 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.623+0000 7ff89c482700 1 -- 172.21.15.92:0/2172755425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 msgr2=0x7ff89410ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.623+0000 7ff89c482700 1 --2- 172.21.15.92:0/2172755425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff89410ab70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8840098a0 tx=0x7ff884009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:45.551 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.623+0000 7ff89c482700 1 -- 172.21.15.92:0/2172755425 shutdown_connections 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.623+0000 7ff89c482700 1 --2- 172.21.15.92:0/2172755425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff89410ab70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.623+0000 7ff89c482700 1 -- 172.21.15.92:0/2172755425 >> 172.21.15.92:0/2172755425 conn(0x7ff894102190 msgr2=0x7ff8941045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.623+0000 7ff89c482700 1 -- 172.21.15.92:0/2172755425 shutdown_connections 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.623+0000 7ff89c482700 1 -- 172.21.15.92:0/2172755425 wait complete. 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.623+0000 7ff89c482700 1 Processor -- start 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.623+0000 7ff89c482700 1 -- start start 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff89c482700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff894108000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff89c482700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff89410c110 con 0x7ff894108780 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff89a21e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff894108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff89a21e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff894108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44794/0 (socket says 172.21.15.92:44794) 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff89a21e700 1 -- 172.21.15.92:0/1484588501 learned_addr learned my addr 172.21.15.92:0/1484588501 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff89a21e700 1 -- 172.21.15.92:0/1484588501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff894108540 con 0x7ff894108780 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff89a21e700 1 --2- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff894108000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff884011b60 tx=0x7ff884011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:45.552 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff88b7fe700 1 -- 172.21.15.92:0/1484588501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff884003b60 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff88b7fe700 1 -- 172.21.15.92:0/1484588501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff88401a910 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff894106730 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff88b7fe700 1 -- 172.21.15.92:0/1484588501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff884024930 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.624+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff894106bf0 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.625+0000 7ff88b7fe700 1 -- 172.21.15.92:0/1484588501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7ff884022020 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.625+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff894063940 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.625+0000 7ff88b7fe700 1 --2- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff880038fd0 0x7ff88003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.625+0000 7ff88b7fe700 1 -- 172.21.15.92:0/1484588501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff88404d140 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.626+0000 7ff899a1d700 1 --2- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff880038fd0 0x7ff88003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.626+0000 7ff899a1d700 1 --2- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff880038fd0 0x7ff88003b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff890006fd0 tx=0x7ff890006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.630+0000 7ff88b7fe700 1 -- 172.21.15.92:0/1484588501 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff88402d770 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.837+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff894106e70 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.838+0000 7ff88b7fe700 1 -- 172.21.15.92:0/1484588501 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff88401aac0 con 0x7ff894108780 2024-03-19T01:43:45.553 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff880038fd0 msgr2=0x7ff88003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:45.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 --2- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff880038fd0 0x7ff88003b490 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff890006fd0 tx=0x7ff890006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:45.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 msgr2=0x7ff894108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:45.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 --2- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff894108000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff884011b60 tx=0x7ff884011c40 comp rx=0 tx=0).stop 2024-03-19T01:43:45.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 shutdown_connections 2024-03-19T01:43:45.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 --2- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff880038fd0 0x7ff88003b490 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:45.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 --2- 172.21.15.92:0/1484588501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff894108780 0x7ff894108000 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:45.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 >> 172.21.15.92:0/1484588501 conn(0x7ff894102190 msgr2=0x7ff894102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:45.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 shutdown_connections 2024-03-19T01:43:45.554 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:44.840+0000 7ff89c482700 1 -- 172.21.15.92:0/1484588501 wait complete. 2024-03-19T01:43:45.557 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-19T01:43:45.557 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout disabled 2024-03-19T01:43:45.565 INFO:teuthology.orchestra.run.smithi092.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-19T01:43:45.565 INFO:teuthology.orchestra.run.smithi092.stdout:systemctl: stdout inactive 2024-03-19T01:43:45.565 INFO:teuthology.orchestra.run.smithi092.stdout:firewalld.service is not enabled 2024-03-19T01:43:45.565 INFO:teuthology.orchestra.run.smithi092.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-19T01:43:45.566 INFO:teuthology.orchestra.run.smithi092.stdout:Ceph Dashboard is now available at: 2024-03-19T01:43:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: URL: https://smithi092.front.sepia.ceph.com:8443/ 2024-03-19T01:43:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: User: admin 2024-03-19T01:43:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: Password: ampxo08kmx 2024-03-19T01:43:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:45.567 INFO:teuthology.orchestra.run.smithi092.stdout:Saving cluster configuration to /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config directory 2024-03-19T01:43:45.567 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling autotune for osd_memory_target 2024-03-19T01:43:45.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:45 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1484588501' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-19T01:43:46.667 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:46 smithi092 ceph-mon[62574]: mgrmap e13: x(active, since 6s) 2024-03-19T01:43:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.457+0000 7f1b7bb21700 1 Processor -- start 2024-03-19T01:43:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.458+0000 7f1b7bb21700 1 -- start start 2024-03-19T01:43:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.458+0000 7f1b7bb21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7407c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.458+0000 7f1b7bb21700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7407c850 con 0x7f1b7407de20 2024-03-19T01:43:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.458+0000 7f1b798bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7407c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.458+0000 7f1b798bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7407c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44798/0 (socket says 172.21.15.92:44798) 2024-03-19T01:43:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.458+0000 7f1b798bd700 1 -- 172.21.15.92:0/1130636235 learned_addr learned my addr 172.21.15.92:0/1130636235 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:47.478 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.458+0000 7f1b798bd700 1 -- 172.21.15.92:0/1130636235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7407c9d0 con 0x7f1b7407de20 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.458+0000 7f1b798bd700 1 --2- 172.21.15.92:0/1130636235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7407c280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1b640098a0 tx=0x7f1b64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727f4c0e3cb92562 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b788bb700 1 -- 172.21.15.92:0/1130636235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b64004160 con 0x7f1b7407de20 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b788bb700 1 -- 172.21.15.92:0/1130636235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64004300 con 0x7f1b7407de20 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b788bb700 1 -- 172.21.15.92:0/1130636235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b64011b20 con 0x7f1b7407de20 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b7bb21700 1 -- 172.21.15.92:0/1130636235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 msgr2=0x7f1b7407c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b7bb21700 1 --2- 172.21.15.92:0/1130636235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7407c280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1b640098a0 tx=0x7f1b64009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b7bb21700 1 -- 172.21.15.92:0/1130636235 shutdown_connections 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b7bb21700 1 --2- 172.21.15.92:0/1130636235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7407c280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b7bb21700 1 -- 172.21.15.92:0/1130636235 >> 172.21.15.92:0/1130636235 conn(0x7f1b74103200 msgr2=0x7f1b74105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b7bb21700 1 -- 172.21.15.92:0/1130636235 shutdown_connections 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.459+0000 7f1b7bb21700 1 -- 172.21.15.92:0/1130636235 wait complete. 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.460+0000 7f1b7bb21700 1 Processor -- start 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.460+0000 7f1b7bb21700 1 -- start start 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.460+0000 7f1b7bb21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:47.479 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.460+0000 7f1b7bb21700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7410e2c0 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.460+0000 7f1b798bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.460+0000 7f1b798bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44802/0 (socket says 172.21.15.92:44802) 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.460+0000 7f1b798bd700 1 -- 172.21.15.92:0/4093702805 learned_addr learned my addr 172.21.15.92:0/4093702805 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.460+0000 7f1b798bd700 1 -- 172.21.15.92:0/4093702805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7419e350 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.461+0000 7f1b798bd700 1 --2- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7419de10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b640116a0 tx=0x7f1b64011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.461+0000 7f1b6affd700 1 -- 172.21.15.92:0/4093702805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b64003a40 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.461+0000 7f1b6affd700 1 -- 172.21.15.92:0/4093702805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b64003be0 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.461+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7419e5e0 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.461+0000 7f1b6affd700 1 -- 172.21.15.92:0/4093702805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b6401a720 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.461+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7419ea40 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.462+0000 7f1b6affd700 1 -- 172.21.15.92:0/4093702805 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f1b6401a9d0 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.462+0000 7f1b6affd700 1 --2- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1b60039020 0x7f1b6003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.462+0000 7f1b6affd700 1 -- 172.21.15.92:0/4093702805 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1b6404d590 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.462+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b74063940 con 0x7f1b7407de20 2024-03-19T01:43:47.480 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.462+0000 7f1b790bc700 1 --2- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1b60039020 0x7f1b6003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.462+0000 7f1b790bc700 1 --2- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1b60039020 0x7f1b6003b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006fd0 tx=0x7f1b70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.467+0000 7f1b6affd700 1 -- 172.21.15.92:0/4093702805 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b64011e20 con 0x7f1b7407de20 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.666+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f1b741a1490 con 0x7f1b7407de20 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.667+0000 7f1b6affd700 1 -- 172.21.15.92:0/4093702805 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f1b64032020 con 0x7f1b7407de20 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1b60039020 msgr2=0x7f1b6003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 --2- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1b60039020 0x7f1b6003b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006fd0 tx=0x7f1b70006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 msgr2=0x7f1b7419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 --2- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7419de10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b640116a0 tx=0x7f1b64011ba0 comp rx=0 tx=0).stop 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 shutdown_connections 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 --2- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1b60039020 0x7f1b6003b4e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 --2- 172.21.15.92:0/4093702805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7407de20 0x7f1b7419de10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 >> 172.21.15.92:0/4093702805 conn(0x7f1b74103200 msgr2=0x7f1b74103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 shutdown_connections 2024-03-19T01:43:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:46.669+0000 7f1b7bb21700 1 -- 172.21.15.92:0/4093702805 wait complete. 2024-03-19T01:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.931+0000 7f7e39253700 1 Processor -- start 2024-03-19T01:43:48.868 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.931+0000 7f7e39253700 1 -- start start 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.931+0000 7f7e39253700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3410ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.932+0000 7f7e39253700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3407bdd0 con 0x7f7e34108770 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.932+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3410ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.932+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3410ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44804/0 (socket says 172.21.15.92:44804) 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.932+0000 7f7e32d9d700 1 -- 172.21.15.92:0/781126455 learned_addr learned my addr 172.21.15.92:0/781126455 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.932+0000 7f7e32d9d700 1 -- 172.21.15.92:0/781126455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3410b0a0 con 0x7f7e34108770 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.932+0000 7f7e32d9d700 1 --2- 172.21.15.92:0/781126455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3410ab60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e240098a0 tx=0x7f7e24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6531ec36cc4fdb4c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.932+0000 7f7e31d9b700 1 -- 172.21.15.92:0/781126455 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e24004160 con 0x7f7e34108770 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.932+0000 7f7e31d9b700 1 -- 172.21.15.92:0/781126455 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e24004300 con 0x7f7e34108770 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.932+0000 7f7e31d9b700 1 -- 172.21.15.92:0/781126455 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e24011b20 con 0x7f7e34108770 2024-03-19T01:43:48.869 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.933+0000 7f7e39253700 1 -- 172.21.15.92:0/781126455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 msgr2=0x7f7e3410ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.933+0000 7f7e39253700 1 --2- 172.21.15.92:0/781126455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3410ab60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e240098a0 tx=0x7f7e24009bb0 comp rx=0 tx=0).stop 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.933+0000 7f7e39253700 1 -- 172.21.15.92:0/781126455 shutdown_connections 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.933+0000 7f7e39253700 1 --2- 172.21.15.92:0/781126455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3410ab60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.933+0000 7f7e39253700 1 -- 172.21.15.92:0/781126455 >> 172.21.15.92:0/781126455 conn(0x7f7e34102140 msgr2=0x7f7e34104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.933+0000 7f7e39253700 1 -- 172.21.15.92:0/781126455 shutdown_connections 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.933+0000 7f7e39253700 1 -- 172.21.15.92:0/781126455 wait complete. 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.934+0000 7f7e39253700 1 Processor -- start 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.934+0000 7f7e39253700 1 -- start start 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.934+0000 7f7e39253700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.934+0000 7f7e39253700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3410c0b0 con 0x7f7e34108770 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.934+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.934+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44818/0 (socket says 172.21.15.92:44818) 2024-03-19T01:43:48.870 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.934+0000 7f7e32d9d700 1 -- 172.21.15.92:0/2564371395 learned_addr learned my addr 172.21.15.92:0/2564371395 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.934+0000 7f7e32d9d700 1 -- 172.21.15.92:0/2564371395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3419c0d0 con 0x7f7e34108770 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.935+0000 7f7e32d9d700 1 --2- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3419bb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e240116a0 tx=0x7f7e24011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.935+0000 7f7e1bfff700 1 -- 172.21.15.92:0/2564371395 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e24003a40 con 0x7f7e34108770 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.935+0000 7f7e1bfff700 1 -- 172.21.15.92:0/2564371395 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e24003be0 con 0x7f7e34108770 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.935+0000 7f7e1bfff700 1 -- 172.21.15.92:0/2564371395 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e2401a750 con 0x7f7e34108770 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.935+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3419c360 con 0x7f7e34108770 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.935+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3419c7c0 con 0x7f7e34108770 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.936+0000 7f7e1bfff700 1 -- 172.21.15.92:0/2564371395 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f7e2401aa00 con 0x7f7e34108770 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.936+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e34063940 con 0x7f7e34108770 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.936+0000 7f7e1bfff700 1 --2- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e1c039020 0x7f7e1c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.936+0000 7f7e1bfff700 1 -- 172.21.15.92:0/2564371395 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e2404d900 con 0x7f7e34108770 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.936+0000 7f7e3259c700 1 --2- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e1c039020 0x7f7e1c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.936+0000 7f7e3259c700 1 --2- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e1c039020 0x7f7e1c03b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e28006fd0 tx=0x7f7e28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:48.871 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:47.941+0000 7f7e1bfff700 1 -- 172.21.15.92:0/2564371395 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7e24011e20 con 0x7f7e34108770 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.241+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f7e3419f190 con 0x7f7e34108770 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.243+0000 7f7e1bfff700 1 -- 172.21.15.92:0/2564371395 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f7e24019c70 con 0x7f7e34108770 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.245+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e1c039020 msgr2=0x7f7e1c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.245+0000 7f7e39253700 1 --2- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e1c039020 0x7f7e1c03b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e28006fd0 tx=0x7f7e28006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.246+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 msgr2=0x7f7e3419bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.246+0000 7f7e39253700 1 --2- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3419bb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e240116a0 tx=0x7f7e24011ba0 comp rx=0 tx=0).stop 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.246+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 shutdown_connections 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.246+0000 7f7e39253700 1 --2- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e1c039020 0x7f7e1c03b4e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.246+0000 7f7e39253700 1 --2- 172.21.15.92:0/2564371395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e34108770 0x7f7e3419bb90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.246+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 >> 172.21.15.92:0/2564371395 conn(0x7f7e34102140 msgr2=0x7f7e34102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.246+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 shutdown_connections 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr 2024-03-19T01:43:48.246+0000 7f7e39253700 1 -- 172.21.15.92:0/2564371395 wait complete. 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-19T01:43:48.872 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout:Or, if you are only running a single cluster on this host: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: ceph telemetry on 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout:For more information see: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:43:48.873 INFO:teuthology.orchestra.run.smithi092.stdout:Bootstrap complete. 2024-03-19T01:43:48.912 INFO:tasks.cephadm:Fetching config... 2024-03-19T01:43:48.913 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:43:48.913 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-19T01:43:48.929 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-19T01:43:48.929 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:43:48.929 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-19T01:43:48.984 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-19T01:43:48.984 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:43:48.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-19T01:43:49.051 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-19T01:43:49.051 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:43:49.051 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-19T01:43:49.106 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-19T01:43:49.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEk/umEnrSLoAXRE8bb9OUNzE09wfce/czVvxSzfTikpPcyy9vsloqk3OUHgJhV5n6nhoI5TVkWzVqwJUlubn6U6JFJIFwoSzUc99e5tHCMcFNqfPxBPDs80n1KzCL3cJbCatFuNCQ7Yz7TMmC/1cOdrXButX86U7bgs2NJqPaxVaDGf5JlLwUj3ViaD2EcSba5cJFyYt7cj4/i5xkFC9SEv5uOGf6HLEFXKuhxgN9KwUOEg6lqMIl9t9QP2aLkfgrx63Zc81eSBo2P1/Ic0FbrD38YMZjzAUzkZa6hGJ7ofBC3PZ49Kai6Qo5nO/QyhrX+YmNE7TwstGdxGduNF1vMTDHxKIsdtV9Bd1VqvlatWR8mWAG0khY9fsRhRw4PAOzWxjIAC0ybeYdsEY9DESvGDfaDK/TO4jBq73ra9zJnCHdiw6FGdSl02O3iiZLcT1v4Nr7CJaPrzkxblhkhxn/nRQ1xty2LF/BvZ6uTu+9WcWDShJxsGa+gfZv9/mO4tc= ceph-d1e857f4-e591-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-19T01:43:49.202 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEk/umEnrSLoAXRE8bb9OUNzE09wfce/czVvxSzfTikpPcyy9vsloqk3OUHgJhV5n6nhoI5TVkWzVqwJUlubn6U6JFJIFwoSzUc99e5tHCMcFNqfPxBPDs80n1KzCL3cJbCatFuNCQ7Yz7TMmC/1cOdrXButX86U7bgs2NJqPaxVaDGf5JlLwUj3ViaD2EcSba5cJFyYt7cj4/i5xkFC9SEv5uOGf6HLEFXKuhxgN9KwUOEg6lqMIl9t9QP2aLkfgrx63Zc81eSBo2P1/Ic0FbrD38YMZjzAUzkZa6hGJ7ofBC3PZ49Kai6Qo5nO/QyhrX+YmNE7TwstGdxGduNF1vMTDHxKIsdtV9Bd1VqvlatWR8mWAG0khY9fsRhRw4PAOzWxjIAC0ybeYdsEY9DESvGDfaDK/TO4jBq73ra9zJnCHdiw6FGdSl02O3iiZLcT1v4Nr7CJaPrzkxblhkhxn/nRQ1xty2LF/BvZ6uTu+9WcWDShJxsGa+gfZv9/mO4tc= ceph-d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:43:49.216 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEk/umEnrSLoAXRE8bb9OUNzE09wfce/czVvxSzfTikpPcyy9vsloqk3OUHgJhV5n6nhoI5TVkWzVqwJUlubn6U6JFJIFwoSzUc99e5tHCMcFNqfPxBPDs80n1KzCL3cJbCatFuNCQ7Yz7TMmC/1cOdrXButX86U7bgs2NJqPaxVaDGf5JlLwUj3ViaD2EcSba5cJFyYt7cj4/i5xkFC9SEv5uOGf6HLEFXKuhxgN9KwUOEg6lqMIl9t9QP2aLkfgrx63Zc81eSBo2P1/Ic0FbrD38YMZjzAUzkZa6hGJ7ofBC3PZ49Kai6Qo5nO/QyhrX+YmNE7TwstGdxGduNF1vMTDHxKIsdtV9Bd1VqvlatWR8mWAG0khY9fsRhRw4PAOzWxjIAC0ybeYdsEY9DESvGDfaDK/TO4jBq73ra9zJnCHdiw6FGdSl02O3iiZLcT1v4Nr7CJaPrzkxblhkhxn/nRQ1xty2LF/BvZ6uTu+9WcWDShJxsGa+gfZv9/mO4tc= ceph-d1e857f4-e591-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-19T01:43:49.260 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEk/umEnrSLoAXRE8bb9OUNzE09wfce/czVvxSzfTikpPcyy9vsloqk3OUHgJhV5n6nhoI5TVkWzVqwJUlubn6U6JFJIFwoSzUc99e5tHCMcFNqfPxBPDs80n1KzCL3cJbCatFuNCQ7Yz7TMmC/1cOdrXButX86U7bgs2NJqPaxVaDGf5JlLwUj3ViaD2EcSba5cJFyYt7cj4/i5xkFC9SEv5uOGf6HLEFXKuhxgN9KwUOEg6lqMIl9t9QP2aLkfgrx63Zc81eSBo2P1/Ic0FbrD38YMZjzAUzkZa6hGJ7ofBC3PZ49Kai6Qo5nO/QyhrX+YmNE7TwstGdxGduNF1vMTDHxKIsdtV9Bd1VqvlatWR8mWAG0khY9fsRhRw4PAOzWxjIAC0ybeYdsEY9DESvGDfaDK/TO4jBq73ra9zJnCHdiw6FGdSl02O3iiZLcT1v4Nr7CJaPrzkxblhkhxn/nRQ1xty2LF/BvZ6uTu+9WcWDShJxsGa+gfZv9/mO4tc= ceph-d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:43:49.275 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEk/umEnrSLoAXRE8bb9OUNzE09wfce/czVvxSzfTikpPcyy9vsloqk3OUHgJhV5n6nhoI5TVkWzVqwJUlubn6U6JFJIFwoSzUc99e5tHCMcFNqfPxBPDs80n1KzCL3cJbCatFuNCQ7Yz7TMmC/1cOdrXButX86U7bgs2NJqPaxVaDGf5JlLwUj3ViaD2EcSba5cJFyYt7cj4/i5xkFC9SEv5uOGf6HLEFXKuhxgN9KwUOEg6lqMIl9t9QP2aLkfgrx63Zc81eSBo2P1/Ic0FbrD38YMZjzAUzkZa6hGJ7ofBC3PZ49Kai6Qo5nO/QyhrX+YmNE7TwstGdxGduNF1vMTDHxKIsdtV9Bd1VqvlatWR8mWAG0khY9fsRhRw4PAOzWxjIAC0ybeYdsEY9DESvGDfaDK/TO4jBq73ra9zJnCHdiw6FGdSl02O3iiZLcT1v4Nr7CJaPrzkxblhkhxn/nRQ1xty2LF/BvZ6uTu+9WcWDShJxsGa+gfZv9/mO4tc= ceph-d1e857f4-e591-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-19T01:43:49.321 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEk/umEnrSLoAXRE8bb9OUNzE09wfce/czVvxSzfTikpPcyy9vsloqk3OUHgJhV5n6nhoI5TVkWzVqwJUlubn6U6JFJIFwoSzUc99e5tHCMcFNqfPxBPDs80n1KzCL3cJbCatFuNCQ7Yz7TMmC/1cOdrXButX86U7bgs2NJqPaxVaDGf5JlLwUj3ViaD2EcSba5cJFyYt7cj4/i5xkFC9SEv5uOGf6HLEFXKuhxgN9KwUOEg6lqMIl9t9QP2aLkfgrx63Zc81eSBo2P1/Ic0FbrD38YMZjzAUzkZa6hGJ7ofBC3PZ49Kai6Qo5nO/QyhrX+YmNE7TwstGdxGduNF1vMTDHxKIsdtV9Bd1VqvlatWR8mWAG0khY9fsRhRw4PAOzWxjIAC0ybeYdsEY9DESvGDfaDK/TO4jBq73ra9zJnCHdiw6FGdSl02O3iiZLcT1v4Nr7CJaPrzkxblhkhxn/nRQ1xty2LF/BvZ6uTu+9WcWDShJxsGa+gfZv9/mO4tc= ceph-d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:43:49.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-19T01:43:49.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:49 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2564371395' entity='client.admin' 2024-03-19T01:43:49.579 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:43:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.806+0000 7f344f89b700 1 -- 172.21.15.92:0/4179315554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 msgr2=0x7f3448102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:51.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.806+0000 7f344f89b700 1 --2- 172.21.15.92:0/4179315554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 0x7f3448102180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3444009910 tx=0x7f3444009c20 comp rx=0 tx=0).stop 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.806+0000 7f344f89b700 1 -- 172.21.15.92:0/4179315554 shutdown_connections 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.806+0000 7f344f89b700 1 --2- 172.21.15.92:0/4179315554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 0x7f3448102180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.806+0000 7f344f89b700 1 -- 172.21.15.92:0/4179315554 >> 172.21.15.92:0/4179315554 conn(0x7f34480fd300 msgr2=0x7f34480ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.806+0000 7f344f89b700 1 -- 172.21.15.92:0/4179315554 shutdown_connections 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.807+0000 7f344f89b700 1 -- 172.21.15.92:0/4179315554 wait complete. 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.807+0000 7f344f89b700 1 Processor -- start 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.807+0000 7f344f89b700 1 -- start start 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.807+0000 7f344f89b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 0x7f3448197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.807+0000 7f344f89b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3448107df0 con 0x7f3448101d60 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.807+0000 7f344d637700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 0x7f3448197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.807+0000 7f344d637700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 0x7f3448197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44836/0 (socket says 172.21.15.92:44836) 2024-03-19T01:43:51.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.807+0000 7f344d637700 1 -- 172.21.15.92:0/4266579649 learned_addr learned my addr 172.21.15.92:0/4266579649 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.808+0000 7f344d637700 1 -- 172.21.15.92:0/4266579649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34481979b0 con 0x7f3448101d60 2024-03-19T01:43:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.808+0000 7f344d637700 1 --2- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 0x7f3448197470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3444006010 tx=0x7f3444005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.808+0000 7f343e7fc700 1 -- 172.21.15.92:0/4266579649 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3444027070 con 0x7f3448101d60 2024-03-19T01:43:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.808+0000 7f343e7fc700 1 -- 172.21.15.92:0/4266579649 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344400be40 con 0x7f3448101d60 2024-03-19T01:43:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.808+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3448197c40 con 0x7f3448101d60 2024-03-19T01:43:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.808+0000 7f343e7fc700 1 -- 172.21.15.92:0/4266579649 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f344400f460 con 0x7f3448101d60 2024-03-19T01:43:51.810 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.808+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344819a930 con 0x7f3448101d60 2024-03-19T01:43:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.809+0000 7f343e7fc700 1 -- 172.21.15.92:0/4266579649 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f344402a070 con 0x7f3448101d60 2024-03-19T01:43:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.809+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3448108570 con 0x7f3448101d60 2024-03-19T01:43:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.809+0000 7f343e7fc700 1 --2- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3434039020 0x7f343403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.809+0000 7f343e7fc700 1 -- 172.21.15.92:0/4266579649 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f344401e030 con 0x7f3448101d60 2024-03-19T01:43:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.809+0000 7f344ce36700 1 --2- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3434039020 0x7f343403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:51.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.809+0000 7f344ce36700 1 --2- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3434039020 0x7f343403b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3438006fd0 tx=0x7f3438006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:51.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:51.814+0000 7f343e7fc700 1 -- 172.21.15.92:0/4266579649 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3444013070 con 0x7f3448101d60 2024-03-19T01:43:52.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.016+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f344802dcc0 con 0x7f3448101d60 2024-03-19T01:43:52.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.018+0000 7f343e7fc700 1 -- 172.21.15.92:0/4266579649 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3444031020 con 0x7f3448101d60 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3434039020 msgr2=0x7f343403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 --2- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3434039020 0x7f343403b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3438006fd0 tx=0x7f3438006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 msgr2=0x7f3448197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 --2- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 0x7f3448197470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3444006010 tx=0x7f3444005050 comp rx=0 tx=0).stop 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 shutdown_connections 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 --2- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3434039020 0x7f343403b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 --2- 172.21.15.92:0/4266579649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3448101d60 0x7f3448197470 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 >> 172.21.15.92:0/4266579649 conn(0x7f34480fd300 msgr2=0x7f34480fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 shutdown_connections 2024-03-19T01:43:52.022 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:52.021+0000 7f344f89b700 1 -- 172.21.15.92:0/4266579649 wait complete. 2024-03-19T01:43:52.861 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-19T01:43:52.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-19T01:43:53.161 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:53 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4266579649' entity='client.admin' 2024-03-19T01:43:53.534 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:43:54.650 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:54.651 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:54.651 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:43:54.651 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:54.651 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:54.651 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.133+0000 7f10e5f8f700 1 -- 172.21.15.92:0/3463735636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 msgr2=0x7f10e0102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.133+0000 7f10e5f8f700 1 --2- 172.21.15.92:0/3463735636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 0x7f10e0102630 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009910 tx=0x7f10d0009c20 comp rx=0 tx=0).stop 2024-03-19T01:43:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.133+0000 7f10e5f8f700 1 -- 172.21.15.92:0/3463735636 shutdown_connections 2024-03-19T01:43:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.133+0000 7f10e5f8f700 1 --2- 172.21.15.92:0/3463735636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 0x7f10e0102630 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.133+0000 7f10e5f8f700 1 -- 172.21.15.92:0/3463735636 >> 172.21.15.92:0/3463735636 conn(0x7f10e00fb9c0 msgr2=0x7f10e00fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.133+0000 7f10e5f8f700 1 -- 172.21.15.92:0/3463735636 shutdown_connections 2024-03-19T01:43:55.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.133+0000 7f10e5f8f700 1 -- 172.21.15.92:0/3463735636 wait complete. 2024-03-19T01:43:55.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.134+0000 7f10e5f8f700 1 Processor -- start 2024-03-19T01:43:55.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.134+0000 7f10e5f8f700 1 -- start start 2024-03-19T01:43:55.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.134+0000 7f10e5f8f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 0x7f10e0197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:55.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.134+0000 7f10e5f8f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0106d10 con 0x7f10e0102210 2024-03-19T01:43:55.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.134+0000 7f10df7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 0x7f10e0197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:55.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.134+0000 7f10df7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 0x7f10e0197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33296/0 (socket says 172.21.15.92:33296) 2024-03-19T01:43:55.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.134+0000 7f10df7fe700 1 -- 172.21.15.92:0/463377680 learned_addr learned my addr 172.21.15.92:0/463377680 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:55.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.135+0000 7f10df7fe700 1 -- 172.21.15.92:0/463377680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e0197970 con 0x7f10e0102210 2024-03-19T01:43:55.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.135+0000 7f10df7fe700 1 --2- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 0x7f10e0197430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10d0006010 tx=0x7f10d0005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.135+0000 7f10dcff9700 1 -- 172.21.15.92:0/463377680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10d0027070 con 0x7f10e0102210 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.135+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e0197c00 con 0x7f10e0102210 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.135+0000 7f10dcff9700 1 -- 172.21.15.92:0/463377680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d000be40 con 0x7f10e0102210 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.135+0000 7f10dcff9700 1 -- 172.21.15.92:0/463377680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10d000f460 con 0x7f10e0102210 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.135+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e019a8f0 con 0x7f10e0102210 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.136+0000 7f10dcff9700 1 -- 172.21.15.92:0/463377680 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f10d002a070 con 0x7f10e0102210 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.136+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e0107490 con 0x7f10e0102210 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.136+0000 7f10dcff9700 1 --2- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f10c8038fc0 0x7f10c803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.136+0000 7f10dcff9700 1 -- 172.21.15.92:0/463377680 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f10d001e030 con 0x7f10e0102210 2024-03-19T01:43:55.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.136+0000 7f10deffd700 1 --2- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f10c8038fc0 0x7f10c803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:55.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.136+0000 7f10deffd700 1 --2- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f10c8038fc0 0x7f10c803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:55.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.141+0000 7f10dcff9700 1 -- 172.21.15.92:0/463377680 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f10d0013070 con 0x7f10e0102210 2024-03-19T01:43:55.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.343+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f10e002dcc0 con 0x7f10c8038fc0 2024-03-19T01:43:55.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.346+0000 7f10dcff9700 1 -- 172.21.15.92:0/463377680 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10e002dcc0 con 0x7f10c8038fc0 2024-03-19T01:43:55.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f10c8038fc0 msgr2=0x7f10c803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:55.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 --2- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f10c8038fc0 0x7f10c803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4006e40 comp rx=0 tx=0).stop 2024-03-19T01:43:55.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 msgr2=0x7f10e0197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:55.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 --2- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 0x7f10e0197430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10d0006010 tx=0x7f10d0005050 comp rx=0 tx=0).stop 2024-03-19T01:43:55.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 shutdown_connections 2024-03-19T01:43:55.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 --2- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f10c8038fc0 0x7f10c803b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:55.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 --2- 172.21.15.92:0/463377680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10e0102210 0x7f10e0197430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:55.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 >> 172.21.15.92:0/463377680 conn(0x7f10e00fb9c0 msgr2=0x7f10e0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:55.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 shutdown_connections 2024-03-19T01:43:55.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:55.349+0000 7f10e5f8f700 1 -- 172.21.15.92:0/463377680 wait complete. 2024-03-19T01:43:56.168 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-03-19T01:43:56.169 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:43:56.169 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-03-19T01:43:56.189 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:43:56.189 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:43:56.246 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-03-19T01:43:56.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch host add smithi144 2024-03-19T01:43:56.481 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:43:56.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:43:56.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:56.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:43:56.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:43:56.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:43:56.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: Updating smithi092:/etc/ceph/ceph.conf 2024-03-19T01:43:56.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: Updating smithi092:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:43:56.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:56.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:56.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:43:57.533 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:57 smithi092 ceph-mon[62574]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:43:57.533 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:57 smithi092 ceph-mon[62574]: Updating smithi092:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:43:57.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 -- 172.21.15.92:0/1632918494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 msgr2=0x7f98a4101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 --2- 172.21.15.92:0/1632918494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 0x7f98a4101230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009910 tx=0x7f98a0009c20 comp rx=0 tx=0).stop 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 -- 172.21.15.92:0/1632918494 shutdown_connections 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 --2- 172.21.15.92:0/1632918494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 0x7f98a4101230 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 -- 172.21.15.92:0/1632918494 >> 172.21.15.92:0/1632918494 conn(0x7f98a40fc700 msgr2=0x7f98a40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 -- 172.21.15.92:0/1632918494 shutdown_connections 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 -- 172.21.15.92:0/1632918494 wait complete. 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 Processor -- start 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 -- start start 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 0x7f98a41028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.997+0000 7f98abe1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410b220 con 0x7f98a4100e50 2024-03-19T01:43:57.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f98a9bb6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 0x7f98a41028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:58.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f98a9bb6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 0x7f98a41028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33312/0 (socket says 172.21.15.92:33312) 2024-03-19T01:43:58.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f98a9bb6700 1 -- 172.21.15.92:0/3147008198 learned_addr learned my addr 172.21.15.92:0/3147008198 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:43:58.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f98a9bb6700 1 -- 172.21.15.92:0/3147008198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a41047e0 con 0x7f98a4100e50 2024-03-19T01:43:58.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f98a9bb6700 1 --2- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 0x7f98a41028f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f98a0005850 tx=0x7f98a0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:58.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f989affd700 1 -- 172.21.15.92:0/3147008198 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98a0026070 con 0x7f98a4100e50 2024-03-19T01:43:58.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f989affd700 1 -- 172.21.15.92:0/3147008198 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a000bc30 con 0x7f98a4100e50 2024-03-19T01:43:58.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a4102e90 con 0x7f98a4100e50 2024-03-19T01:43:58.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f989affd700 1 -- 172.21.15.92:0/3147008198 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98a0003a20 con 0x7f98a4100e50 2024-03-19T01:43:58.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.998+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a4103350 con 0x7f98a4100e50 2024-03-19T01:43:58.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.999+0000 7f989affd700 1 -- 172.21.15.92:0/3147008198 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f98a0029070 con 0x7f98a4100e50 2024-03-19T01:43:58.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.999+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a4102330 con 0x7f98a4100e50 2024-03-19T01:43:58.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.999+0000 7f989affd700 1 --2- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9890038fc0 0x7f989003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:43:58.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.999+0000 7f989affd700 1 -- 172.21.15.92:0/3147008198 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f98a001e070 con 0x7f98a4100e50 2024-03-19T01:43:58.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:57.999+0000 7f98a93b5700 1 --2- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9890038fc0 0x7f989003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:43:58.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:58.000+0000 7f98a93b5700 1 --2- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9890038fc0 0x7f989003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9894006fd0 tx=0x7f9894006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:43:58.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:58.004+0000 7f989affd700 1 -- 172.21.15.92:0/3147008198 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f98a0013070 con 0x7f98a4100e50 2024-03-19T01:43:58.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:43:58.206+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}) -- 0x7f98a41035e0 con 0x7f9890038fc0 2024-03-19T01:43:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:58 smithi092 ceph-mon[62574]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:00.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:59 smithi092 ceph-mon[62574]: Deploying cephadm binary to smithi144 2024-03-19T01:44:00.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:43:59 smithi092 ceph-mon[62574]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:02.568 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.567+0000 7f989affd700 1 -- 172.21.15.92:0/3147008198 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f98a41035e0 con 0x7f9890038fc0 2024-03-19T01:44:02.568 INFO:teuthology.orchestra.run.smithi092.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-03-19T01:44:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9890038fc0 msgr2=0x7f989003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 --2- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9890038fc0 0x7f989003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9894006fd0 tx=0x7f9894006e40 comp rx=0 tx=0).stop 2024-03-19T01:44:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 msgr2=0x7f98a41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 --2- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 0x7f98a41028f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f98a0005850 tx=0x7f98a0005030 comp rx=0 tx=0).stop 2024-03-19T01:44:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 shutdown_connections 2024-03-19T01:44:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 --2- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9890038fc0 0x7f989003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 --2- 172.21.15.92:0/3147008198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a4100e50 0x7f98a41028f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 >> 172.21.15.92:0/3147008198 conn(0x7f98a40fc700 msgr2=0x7f98a406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 shutdown_connections 2024-03-19T01:44:02.571 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:02.570+0000 7f98abe1a700 1 -- 172.21.15.92:0/3147008198 wait complete. 2024-03-19T01:44:02.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:02 smithi092 ceph-mon[62574]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:03.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-19T01:44:03.505 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:44:03.841 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:03.841 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:03 smithi092 ceph-mon[62574]: Added host smithi144 2024-03-19T01:44:03.842 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:03.842 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:04.863 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:04 smithi092 ceph-mon[62574]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:04.863 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:04 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:05.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.046+0000 7f2e81e48700 1 -- 172.21.15.92:0/4079043369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 msgr2=0x7f2e7c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:05.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.046+0000 7f2e81e48700 1 --2- 172.21.15.92:0/4079043369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 0x7f2e7c102330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009910 tx=0x7f2e70009c20 comp rx=0 tx=0).stop 2024-03-19T01:44:05.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.046+0000 7f2e81e48700 1 -- 172.21.15.92:0/4079043369 shutdown_connections 2024-03-19T01:44:05.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.046+0000 7f2e81e48700 1 --2- 172.21.15.92:0/4079043369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 0x7f2e7c102330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:05.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.046+0000 7f2e81e48700 1 -- 172.21.15.92:0/4079043369 >> 172.21.15.92:0/4079043369 conn(0x7f2e7c0fd820 msgr2=0x7f2e7c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:05.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.046+0000 7f2e81e48700 1 -- 172.21.15.92:0/4079043369 shutdown_connections 2024-03-19T01:44:05.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.046+0000 7f2e81e48700 1 -- 172.21.15.92:0/4079043369 wait complete. 2024-03-19T01:44:05.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.046+0000 7f2e81e48700 1 Processor -- start 2024-03-19T01:44:05.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e81e48700 1 -- start start 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e81e48700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 0x7f2e7c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e81e48700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c075c80 con 0x7f2e7c101f50 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e7b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 0x7f2e7c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e7b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 0x7f2e7c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37252/0 (socket says 172.21.15.92:37252) 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e7b7fe700 1 -- 172.21.15.92:0/1603480908 learned_addr learned my addr 172.21.15.92:0/1603480908 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e7b7fe700 1 -- 172.21.15.92:0/1603480908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7c19fc60 con 0x7f2e7c101f50 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e7b7fe700 1 --2- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 0x7f2e7c19f720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e70005850 tx=0x7f2e70005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e78ff9700 1 -- 172.21.15.92:0/1603480908 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e70026070 con 0x7f2e7c101f50 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7c19ff50 con 0x7f2e7c101f50 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e78ff9700 1 -- 172.21.15.92:0/1603480908 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7000bbd0 con 0x7f2e7c101f50 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e78ff9700 1 -- 172.21.15.92:0/1603480908 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e70003970 con 0x7f2e7c101f50 2024-03-19T01:44:05.049 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.047+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7c1998e0 con 0x7f2e7c101f50 2024-03-19T01:44:05.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.048+0000 7f2e78ff9700 1 -- 172.21.15.92:0/1603480908 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f2e70029070 con 0x7f2e7c101f50 2024-03-19T01:44:05.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.048+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7c103430 con 0x7f2e7c101f50 2024-03-19T01:44:05.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.049+0000 7f2e78ff9700 1 --2- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2e64038fc0 0x7f2e6403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:05.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.049+0000 7f2e78ff9700 1 -- 172.21.15.92:0/1603480908 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e7001e070 con 0x7f2e7c101f50 2024-03-19T01:44:05.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.049+0000 7f2e7affd700 1 --2- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2e64038fc0 0x7f2e6403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:05.051 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.049+0000 7f2e7affd700 1 --2- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2e64038fc0 0x7f2e6403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c006fd0 tx=0x7f2e6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:05.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.054+0000 7f2e78ff9700 1 -- 172.21.15.92:0/1603480908 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2e70013070 con 0x7f2e7c101f50 2024-03-19T01:44:05.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.265+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e7c19ae10 con 0x7f2e64038fc0 2024-03-19T01:44:05.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.267+0000 7f2e78ff9700 1 -- 172.21.15.92:0/1603480908 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f2e7c19ae10 con 0x7f2e64038fc0 2024-03-19T01:44:05.268 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:44:05.268 INFO:teuthology.orchestra.run.smithi092.stdout:[{"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-03-19T01:44:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2e64038fc0 msgr2=0x7f2e6403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 --2- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2e64038fc0 0x7f2e6403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c006fd0 tx=0x7f2e6c006e40 comp rx=0 tx=0).stop 2024-03-19T01:44:05.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 msgr2=0x7f2e7c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 --2- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 0x7f2e7c19f720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e70005850 tx=0x7f2e70005030 comp rx=0 tx=0).stop 2024-03-19T01:44:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 shutdown_connections 2024-03-19T01:44:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 --2- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2e64038fc0 0x7f2e6403b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 --2- 172.21.15.92:0/1603480908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e7c101f50 0x7f2e7c19f720 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 >> 172.21.15.92:0/1603480908 conn(0x7f2e7c0fd820 msgr2=0x7f2e7c106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 shutdown_connections 2024-03-19T01:44:05.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:05.269+0000 7f2e81e48700 1 -- 172.21.15.92:0/1603480908 wait complete. 2024-03-19T01:44:05.909 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2024-03-19T01:44:05.909 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:44:05.909 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2024-03-19T01:44:05.930 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:44:05.930 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:44:05.988 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2024-03-19T01:44:05.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch host add smithi159 2024-03-19T01:44:06.223 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:44:06.440 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:06 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:06.440 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:06 smithi092 ceph-mon[62574]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:44:06.440 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:06 smithi092 ceph-mon[62574]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:07.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.618+0000 7f8bfa69e700 1 -- 172.21.15.92:0/2737590860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 msgr2=0x7f8bf4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:07.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.618+0000 7f8bfa69e700 1 --2- 172.21.15.92:0/2737590860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 0x7f8bf4103280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009910 tx=0x7f8be4009c20 comp rx=0 tx=0).stop 2024-03-19T01:44:07.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.618+0000 7f8bfa69e700 1 -- 172.21.15.92:0/2737590860 shutdown_connections 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.618+0000 7f8bfa69e700 1 --2- 172.21.15.92:0/2737590860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 0x7f8bf4103280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.618+0000 7f8bfa69e700 1 -- 172.21.15.92:0/2737590860 >> 172.21.15.92:0/2737590860 conn(0x7f8bf4077850 msgr2=0x7f8bf40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.618+0000 7f8bfa69e700 1 -- 172.21.15.92:0/2737590860 shutdown_connections 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.618+0000 7f8bfa69e700 1 -- 172.21.15.92:0/2737590860 wait complete. 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.619+0000 7f8bfa69e700 1 Processor -- start 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.619+0000 7f8bfa69e700 1 -- start start 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.619+0000 7f8bfa69e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 0x7f8bf4100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.619+0000 7f8bfa69e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410c390 con 0x7f8bf4102ea0 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 0x7f8bf4100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 0x7f8bf4100b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37260/0 (socket says 172.21.15.92:37260) 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bf3fff700 1 -- 172.21.15.92:0/493928952 learned_addr learned my addr 172.21.15.92:0/493928952 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bf3fff700 1 -- 172.21.15.92:0/493928952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf41029f0 con 0x7f8bf4102ea0 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bf3fff700 1 --2- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 0x7f8bf4100b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8be4006010 tx=0x7f8be4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:07.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bf17fa700 1 -- 172.21.15.92:0/493928952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be4026070 con 0x7f8bf4102ea0 2024-03-19T01:44:07.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bf17fa700 1 -- 172.21.15.92:0/493928952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be400bc30 con 0x7f8bf4102ea0 2024-03-19T01:44:07.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf4101040 con 0x7f8bf4102ea0 2024-03-19T01:44:07.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bf17fa700 1 -- 172.21.15.92:0/493928952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be4003ab0 con 0x7f8bf4102ea0 2024-03-19T01:44:07.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.620+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf4101500 con 0x7f8bf4102ea0 2024-03-19T01:44:07.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.621+0000 7f8bf17fa700 1 -- 172.21.15.92:0/493928952 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f8be4029070 con 0x7f8bf4102ea0 2024-03-19T01:44:07.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.621+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf4069fe0 con 0x7f8bf4102ea0 2024-03-19T01:44:07.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.621+0000 7f8bf17fa700 1 --2- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8bdc039020 0x7f8bdc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:07.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.621+0000 7f8bf17fa700 1 -- 172.21.15.92:0/493928952 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8be401e070 con 0x7f8bf4102ea0 2024-03-19T01:44:07.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.621+0000 7f8bf37fe700 1 --2- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8bdc039020 0x7f8bdc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:07.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.622+0000 7f8bf37fe700 1 --2- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8bdc039020 0x7f8bdc03b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006fd0 tx=0x7f8be8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:07.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.626+0000 7f8bf17fa700 1 -- 172.21.15.92:0/493928952 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8be4013070 con 0x7f8bf4102ea0 2024-03-19T01:44:07.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:07.826+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}) -- 0x7f8bf4101730 con 0x7f8bdc039020 2024-03-19T01:44:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:08 smithi092 ceph-mon[62574]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:08.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:08.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:44:08.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:08.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:09.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:09 smithi092 ceph-mon[62574]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:09.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:09 smithi092 ceph-mon[62574]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-19T01:44:09.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:09 smithi092 ceph-mon[62574]: Updating smithi144:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:09.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:09 smithi092 ceph-mon[62574]: Deploying cephadm binary to smithi159 2024-03-19T01:44:10.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:10 smithi092 ceph-mon[62574]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:44:10.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:10 smithi092 ceph-mon[62574]: Updating smithi144:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:44:10.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:10 smithi092 ceph-mon[62574]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:12.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.085+0000 7f8bf17fa700 1 -- 172.21.15.92:0/493928952 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8bf4101730 con 0x7f8bdc039020 2024-03-19T01:44:12.085 INFO:teuthology.orchestra.run.smithi092.stdout:Added host 'smithi159' with addr '172.21.15.159' 2024-03-19T01:44:12.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8bdc039020 msgr2=0x7f8bdc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:12.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 --2- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8bdc039020 0x7f8bdc03b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006fd0 tx=0x7f8be8006e40 comp rx=0 tx=0).stop 2024-03-19T01:44:12.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 msgr2=0x7f8bf4100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:12.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 --2- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 0x7f8bf4100b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8be4006010 tx=0x7f8be4005050 comp rx=0 tx=0).stop 2024-03-19T01:44:12.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 shutdown_connections 2024-03-19T01:44:12.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 --2- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8bdc039020 0x7f8bdc03b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 --2- 172.21.15.92:0/493928952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf4102ea0 0x7f8bf4100b00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 >> 172.21.15.92:0/493928952 conn(0x7f8bf4077850 msgr2=0x7f8bf4103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 shutdown_connections 2024-03-19T01:44:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:12.088+0000 7f8bfa69e700 1 -- 172.21.15.92:0/493928952 wait complete. 2024-03-19T01:44:12.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:12 smithi092 ceph-mon[62574]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:12.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:12.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:12.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:12.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:12.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:12.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-19T01:44:12.939 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:44:13.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:13 smithi092 ceph-mon[62574]: Added host smithi159 2024-03-19T01:44:13.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:13.725 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.554+0000 7f2600bda700 1 -- 172.21.15.92:0/212075046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 msgr2=0x7f25fc103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.554+0000 7f2600bda700 1 --2- 172.21.15.92:0/212075046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 0x7f25fc103290 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b0a0 tx=0x7f25f400b3b0 comp rx=0 tx=0).stop 2024-03-19T01:44:14.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.554+0000 7f2600bda700 1 -- 172.21.15.92:0/212075046 shutdown_connections 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.554+0000 7f2600bda700 1 --2- 172.21.15.92:0/212075046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 0x7f25fc103290 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.554+0000 7f2600bda700 1 -- 172.21.15.92:0/212075046 >> 172.21.15.92:0/212075046 conn(0x7f25fc0fc6e0 msgr2=0x7f25fc0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.554+0000 7f2600bda700 1 -- 172.21.15.92:0/212075046 shutdown_connections 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.555+0000 7f2600bda700 1 -- 172.21.15.92:0/212075046 wait complete. 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.555+0000 7f2600bda700 1 Processor -- start 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.555+0000 7f2600bda700 1 -- start start 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.555+0000 7f2600bda700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 0x7f25fc100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.555+0000 7f2600bda700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc10c3a0 con 0x7f25fc102eb0 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f25fa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 0x7f25fc100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f25fa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 0x7f25fc100b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33710/0 (socket says 172.21.15.92:33710) 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f25fa59c700 1 -- 172.21.15.92:0/4204243317 learned_addr learned my addr 172.21.15.92:0/4204243317 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f25fa59c700 1 -- 172.21.15.92:0/4204243317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc102a00 con 0x7f25fc102eb0 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f25fa59c700 1 --2- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 0x7f25fc100b10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000f80 tx=0x7f25f400bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:14.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f25eb7fe700 1 -- 172.21.15.92:0/4204243317 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f4020020 con 0x7f25fc102eb0 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc1010b0 con 0x7f25fc102eb0 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f25eb7fe700 1 -- 172.21.15.92:0/4204243317 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f4018040 con 0x7f25fc102eb0 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f25eb7fe700 1 -- 172.21.15.92:0/4204243317 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25f4017070 con 0x7f25fc102eb0 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.556+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc101570 con 0x7f25fc102eb0 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.557+0000 7f25eb7fe700 1 -- 172.21.15.92:0/4204243317 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f25f4023020 con 0x7f25fc102eb0 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.557+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc06aa30 con 0x7f25fc102eb0 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.557+0000 7f25eb7fe700 1 --2- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25e4039010 0x7f25e403b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.557+0000 7f25eb7fe700 1 -- 172.21.15.92:0/4204243317 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f25f400f050 con 0x7f25fc102eb0 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.557+0000 7f25f9d9b700 1 --2- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25e4039010 0x7f25e403b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:14.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.558+0000 7f25f9d9b700 1 --2- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25e4039010 0x7f25e403b4d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006fd0 tx=0x7f25f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:14.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.562+0000 7f25eb7fe700 1 -- 172.21.15.92:0/4204243317 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f25f402a020 con 0x7f25fc102eb0 2024-03-19T01:44:14.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.763+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25fc1020b0 con 0x7f25e4039010 2024-03-19T01:44:14.764 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:14 smithi092 ceph-mon[62574]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:14.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.764+0000 7f25eb7fe700 1 -- 172.21.15.92:0/4204243317 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f25fc1020b0 con 0x7f25e4039010 2024-03-19T01:44:14.765 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:44:14.765 INFO:teuthology.orchestra.run.smithi092.stdout:[{"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25e4039010 msgr2=0x7f25e403b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 --2- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25e4039010 0x7f25e403b4d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006fd0 tx=0x7f25f0006e40 comp rx=0 tx=0).stop 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 msgr2=0x7f25fc100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 --2- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 0x7f25fc100b10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000f80 tx=0x7f25f400bd20 comp rx=0 tx=0).stop 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 shutdown_connections 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 --2- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25e4039010 0x7f25e403b4d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 --2- 172.21.15.92:0/4204243317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25fc102eb0 0x7f25fc100b10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 >> 172.21.15.92:0/4204243317 conn(0x7f25fc0fc6e0 msgr2=0x7f25fc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 shutdown_connections 2024-03-19T01:44:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:14.766+0000 7f2600bda700 1 -- 172.21.15.92:0/4204243317 wait complete. 2024-03-19T01:44:15.463 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-19T01:44:15.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-19T01:44:15.704 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:44:15.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:15.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:15 smithi092 ceph-mon[62574]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:44:15.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:15 smithi092 ceph-mon[62574]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.216+0000 7f4548d83700 1 -- 172.21.15.92:0/3635085285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 msgr2=0x7f4544101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.216+0000 7f4548d83700 1 --2- 172.21.15.92:0/3635085285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 0x7f4544101230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4534009910 tx=0x7f4534009c20 comp rx=0 tx=0).stop 2024-03-19T01:44:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.216+0000 7f4548d83700 1 -- 172.21.15.92:0/3635085285 shutdown_connections 2024-03-19T01:44:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.216+0000 7f4548d83700 1 --2- 172.21.15.92:0/3635085285 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 0x7f4544101230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.216+0000 7f4548d83700 1 -- 172.21.15.92:0/3635085285 >> 172.21.15.92:0/3635085285 conn(0x7f45440fc700 msgr2=0x7f45440feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.216+0000 7f4548d83700 1 -- 172.21.15.92:0/3635085285 shutdown_connections 2024-03-19T01:44:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.216+0000 7f4548d83700 1 -- 172.21.15.92:0/3635085285 wait complete. 2024-03-19T01:44:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.217+0000 7f4548d83700 1 Processor -- start 2024-03-19T01:44:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.217+0000 7f4548d83700 1 -- start start 2024-03-19T01:44:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.217+0000 7f4548d83700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 0x7f454419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.217+0000 7f4548d83700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f454410b220 con 0x7f4544100e50 2024-03-19T01:44:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.217+0000 7f454259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 0x7f454419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.217+0000 7f454259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 0x7f454419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33720/0 (socket says 172.21.15.92:33720) 2024-03-19T01:44:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.217+0000 7f454259c700 1 -- 172.21.15.92:0/1432780990 learned_addr learned my addr 172.21.15.92:0/1432780990 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:44:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.218+0000 7f454259c700 1 -- 172.21.15.92:0/1432780990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454419fbf0 con 0x7f4544100e50 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.218+0000 7f454259c700 1 --2- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 0x7f454419f6b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4534005850 tx=0x7f4534005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.218+0000 7f45337fe700 1 -- 172.21.15.92:0/1432780990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4534026070 con 0x7f4544100e50 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.218+0000 7f45337fe700 1 -- 172.21.15.92:0/1432780990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f453400bbd0 con 0x7f4544100e50 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.218+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454419fee0 con 0x7f4544100e50 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.218+0000 7f45337fe700 1 -- 172.21.15.92:0/1432780990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4534003970 con 0x7f4544100e50 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.218+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4544199830 con 0x7f4544100e50 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.219+0000 7f45337fe700 1 -- 172.21.15.92:0/1432780990 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f4534029070 con 0x7f4544100e50 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.219+0000 7f45337fe700 1 --2- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f452c039020 0x7f452c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.219+0000 7f45337fe700 1 -- 172.21.15.92:0/1432780990 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f453401e070 con 0x7f4544100e50 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.219+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4544102330 con 0x7f4544100e50 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.219+0000 7f4541d9b700 1 --2- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f452c039020 0x7f452c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.219+0000 7f4541d9b700 1 --2- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f452c039020 0x7f452c03b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4538006fd0 tx=0x7f4538006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.224+0000 7f45337fe700 1 -- 172.21.15.92:0/1432780990 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4534030070 con 0x7f4544100e50 2024-03-19T01:44:17.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:17.425+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f4544063940 con 0x7f4544100e50 2024-03-19T01:44:18.427 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.427+0000 7f45337fe700 1 -- 172.21.15.92:0/1432780990 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f4534013070 con 0x7f4544100e50 2024-03-19T01:44:18.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f452c039020 msgr2=0x7f452c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 --2- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f452c039020 0x7f452c03b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4538006fd0 tx=0x7f4538006e40 comp rx=0 tx=0).stop 2024-03-19T01:44:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 msgr2=0x7f454419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 --2- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 0x7f454419f6b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4534005850 tx=0x7f4534005030 comp rx=0 tx=0).stop 2024-03-19T01:44:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 shutdown_connections 2024-03-19T01:44:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 --2- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f452c039020 0x7f452c03b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 --2- 172.21.15.92:0/1432780990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4544100e50 0x7f454419f6b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 >> 172.21.15.92:0/1432780990 conn(0x7f45440fc700 msgr2=0x7f454406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 shutdown_connections 2024-03-19T01:44:18.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:18.429+0000 7f4548d83700 1 -- 172.21.15.92:0/1432780990 wait complete. 2024-03-19T01:44:18.431 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2024-03-19T01:44:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:18 smithi092 ceph-mon[62574]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:18 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1432780990' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-19T01:44:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:44:18.802 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:18.803 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:19.087 INFO:tasks.cephadm:Adding mon.a on smithi092 2024-03-19T01:44:19.087 INFO:tasks.cephadm:Adding mon.b on smithi144 2024-03-19T01:44:19.087 INFO:tasks.cephadm:Adding mon.c on smithi159 2024-03-19T01:44:19.087 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi092:172.21.15.92=a;smithi144:172.21.15.144=b;smithi159:172.21.15.159=c' 2024-03-19T01:44:19.314 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:19.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:19 smithi092 ceph-mon[62574]: Updating smithi159:/etc/ceph/ceph.conf 2024-03-19T01:44:19.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:19 smithi092 ceph-mon[62574]: Updating smithi159:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:19.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:19 smithi092 ceph-mon[62574]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:44:19.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:19 smithi092 ceph-mon[62574]: Updating smithi159:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:44:19.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:19 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1432780990' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-19T01:44:19.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:19 smithi092 ceph-mon[62574]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:19.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:19.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:19.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:20 smithi092 ceph-mon[62574]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.842+0000 7fed48724700 1 -- 172.21.15.159:0/502681407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 msgr2=0x7fed40102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.842+0000 7fed48724700 1 --2- 172.21.15.159:0/502681407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 0x7fed40102330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009910 tx=0x7fed3c009c20 comp rx=0 tx=0).stop 2024-03-19T01:44:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.842+0000 7fed48724700 1 -- 172.21.15.159:0/502681407 shutdown_connections 2024-03-19T01:44:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.842+0000 7fed48724700 1 --2- 172.21.15.159:0/502681407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 0x7fed40102330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.842+0000 7fed48724700 1 -- 172.21.15.159:0/502681407 >> 172.21.15.159:0/502681407 conn(0x7fed400fd820 msgr2=0x7fed400ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:20.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.842+0000 7fed48724700 1 -- 172.21.15.159:0/502681407 shutdown_connections 2024-03-19T01:44:20.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.842+0000 7fed48724700 1 -- 172.21.15.159:0/502681407 wait complete. 2024-03-19T01:44:20.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.843+0000 7fed48724700 1 Processor -- start 2024-03-19T01:44:20.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.843+0000 7fed48724700 1 -- start start 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.843+0000 7fed48724700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 0x7fed4019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.843+0000 7fed48724700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fed4010c320 con 0x7fed40101f50 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed464c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 0x7fed4019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed464c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 0x7fed4019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.159:59754/0 (socket says 172.21.15.159:59754) 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed464c0700 1 -- 172.21.15.159:0/664106824 learned_addr learned my addr 172.21.15.159:0/664106824 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed464c0700 1 -- 172.21.15.159:0/664106824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4019db60 con 0x7fed40101f50 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed464c0700 1 --2- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 0x7fed4019d620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005850 tx=0x7fed3c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed377fe700 1 -- 172.21.15.159:0/664106824 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed3c026070 con 0x7fed40101f50 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed377fe700 1 -- 172.21.15.159:0/664106824 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c00bbd0 con 0x7fed40101f50 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4019de50 con 0x7fed40101f50 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed377fe700 1 -- 172.21.15.159:0/664106824 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed3c003970 con 0x7fed40101f50 2024-03-19T01:44:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.844+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed401978c0 con 0x7fed40101f50 2024-03-19T01:44:20.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.845+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed40103430 con 0x7fed40101f50 2024-03-19T01:44:20.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.845+0000 7fed377fe700 1 -- 172.21.15.159:0/664106824 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fed3c029070 con 0x7fed40101f50 2024-03-19T01:44:20.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.845+0000 7fed377fe700 1 --2- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fed2c038fc0 0x7fed2c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:20.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.846+0000 7fed377fe700 1 -- 172.21.15.159:0/664106824 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fed3c01e070 con 0x7fed40101f50 2024-03-19T01:44:20.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.846+0000 7fed45cbf700 1 --2- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fed2c038fc0 0x7fed2c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:20.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.846+0000 7fed45cbf700 1 --2- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fed2c038fc0 0x7fed2c03b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fed30006fd0 tx=0x7fed30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:20.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:20.851+0000 7fed377fe700 1 -- 172.21.15.159:0/664106824 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fed3c013070 con 0x7fed40101f50 2024-03-19T01:44:21.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.057+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi144:172.21.15.144=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}) -- 0x7fed40198ce0 con 0x7fed2c038fc0 2024-03-19T01:44:21.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.063+0000 7fed377fe700 1 -- 172.21.15.159:0/664106824 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fed40198ce0 con 0x7fed2c038fc0 2024-03-19T01:44:21.064 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mon update... 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.065+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fed2c038fc0 msgr2=0x7fed2c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.065+0000 7fed48724700 1 --2- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fed2c038fc0 0x7fed2c03b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fed30006fd0 tx=0x7fed30006e40 comp rx=0 tx=0).stop 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.065+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 msgr2=0x7fed4019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.065+0000 7fed48724700 1 --2- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 0x7fed4019d620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed3c005850 tx=0x7fed3c005030 comp rx=0 tx=0).stop 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.065+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 shutdown_connections 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.065+0000 7fed48724700 1 --2- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fed2c038fc0 0x7fed2c03b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.065+0000 7fed48724700 1 --2- 172.21.15.159:0/664106824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed40101f50 0x7fed4019d620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.065+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 >> 172.21.15.159:0/664106824 conn(0x7fed400fd820 msgr2=0x7fed40104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.066+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 shutdown_connections 2024-03-19T01:44:21.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:21.066+0000 7fed48724700 1 -- 172.21.15.159:0/664106824 wait complete. 2024-03-19T01:44:21.670 DEBUG:teuthology.orchestra.run.smithi144:mon.b> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.b.service 2024-03-19T01:44:21.673 DEBUG:teuthology.orchestra.run.smithi159:mon.c> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.c.service 2024-03-19T01:44:21.675 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-19T01:44:21.675 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-19T01:44:22.433 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:22.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi144:172.21.15.144=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:22.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: Saving service mon spec with placement smithi092:172.21.15.92=a;smithi144:172.21.15.144=b;smithi159:172.21.15.159=c;count:3 2024-03-19T01:44:22.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:22.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:22.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:22.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:22.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:22.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:22.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:22.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: Deploying daemon mon.c on smithi159 2024-03-19T01:44:22.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:22 smithi092 ceph-mon[62574]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:24 smithi092 ceph-mon[62574]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:25.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.527+0000 7f5cfccb7700 1 -- 172.21.15.159:0/886461857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 msgr2=0x7f5cf8101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:25.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.527+0000 7f5cfccb7700 1 --2- 172.21.15.159:0/886461857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 0x7f5cf8101220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009910 tx=0x7f5ce8009c20 comp rx=0 tx=0).stop 2024-03-19T01:44:25.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.527+0000 7f5cfccb7700 1 -- 172.21.15.159:0/886461857 shutdown_connections 2024-03-19T01:44:25.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.527+0000 7f5cfccb7700 1 --2- 172.21.15.159:0/886461857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 0x7f5cf8101220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:25.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.527+0000 7f5cfccb7700 1 -- 172.21.15.159:0/886461857 >> 172.21.15.159:0/886461857 conn(0x7f5cf8077850 msgr2=0x7f5cf80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.527+0000 7f5cfccb7700 1 -- 172.21.15.159:0/886461857 shutdown_connections 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.527+0000 7f5cfccb7700 1 -- 172.21.15.159:0/886461857 wait complete. 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.528+0000 7f5cfccb7700 1 Processor -- start 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.528+0000 7f5cfccb7700 1 -- start start 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.528+0000 7f5cfccb7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 0x7f5cf819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.528+0000 7f5cfccb7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf810b210 con 0x7f5cf8100e40 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.528+0000 7f5cf659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 0x7f5cf819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.528+0000 7f5cf659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 0x7f5cf819d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.159:44604/0 (socket says 172.21.15.159:44604) 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.528+0000 7f5cf659c700 1 -- 172.21.15.159:0/2730835935 learned_addr learned my addr 172.21.15.159:0/2730835935 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.529+0000 7f5cf659c700 1 -- 172.21.15.159:0/2730835935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf819daa0 con 0x7f5cf8100e40 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.529+0000 7f5cf659c700 1 --2- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 0x7f5cf819d560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005850 tx=0x7f5ce8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:25.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.529+0000 7f5ce77fe700 1 -- 172.21.15.159:0/2730835935 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ce8026070 con 0x7f5cf8100e40 2024-03-19T01:44:25.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.529+0000 7f5ce77fe700 1 -- 172.21.15.159:0/2730835935 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce800bbd0 con 0x7f5cf8100e40 2024-03-19T01:44:25.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.529+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf819dd90 con 0x7f5cf8100e40 2024-03-19T01:44:25.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.529+0000 7f5ce77fe700 1 -- 172.21.15.159:0/2730835935 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ce8003970 con 0x7f5cf8100e40 2024-03-19T01:44:25.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.529+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf8197800 con 0x7f5cf8100e40 2024-03-19T01:44:25.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.530+0000 7f5ce77fe700 1 -- 172.21.15.159:0/2730835935 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f5ce8029070 con 0x7f5cf8100e40 2024-03-19T01:44:25.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.530+0000 7f5ce77fe700 1 --2- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ce0038fc0 0x7f5ce003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:25.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.530+0000 7f5ce77fe700 1 -- 172.21.15.159:0/2730835935 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5ce801e070 con 0x7f5cf8100e40 2024-03-19T01:44:25.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.530+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf8102320 con 0x7f5cf8100e40 2024-03-19T01:44:25.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.530+0000 7f5cf5d9b700 1 --2- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ce0038fc0 0x7f5ce003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:25.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.531+0000 7f5cf5d9b700 1 --2- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ce0038fc0 0x7f5ce003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5cec006fd0 tx=0x7f5cec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:25.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.539+0000 7f5ce77fe700 1 -- 172.21.15.159:0/2730835935 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5ce8030070 con 0x7f5cf8100e40 2024-03-19T01:44:25.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.838+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5cf80008d0 con 0x7f5cf8100e40 2024-03-19T01:44:25.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.839+0000 7f5ce77fe700 1 -- 172.21.15.159:0/2730835935 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f5ce8013070 con 0x7f5cf8100e40 2024-03-19T01:44:25.841 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:44:25.842 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":1,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","modified":"2024-03-19T01:42:32.859371Z","created":"2024-03-19T01:42:32.859371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-19T01:44:25.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.842+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ce0038fc0 msgr2=0x7f5ce003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:25.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.843+0000 7f5cfccb7700 1 --2- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ce0038fc0 0x7f5ce003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5cec006fd0 tx=0x7f5cec006e40 comp rx=0 tx=0).stop 2024-03-19T01:44:25.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.843+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 msgr2=0x7f5cf819d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:25.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.843+0000 7f5cfccb7700 1 --2- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 0x7f5cf819d560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005850 tx=0x7f5ce8005030 comp rx=0 tx=0).stop 2024-03-19T01:44:25.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.843+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 shutdown_connections 2024-03-19T01:44:25.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.843+0000 7f5cfccb7700 1 --2- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ce0038fc0 0x7f5ce003b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:25.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.843+0000 7f5cfccb7700 1 --2- 172.21.15.159:0/2730835935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cf8100e40 0x7f5cf819d560 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:25.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.843+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 >> 172.21.15.159:0/2730835935 conn(0x7f5cf8077850 msgr2=0x7f5cf8069700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:25.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.843+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 shutdown_connections 2024-03-19T01:44:25.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:25.843+0000 7f5cfccb7700 1 -- 172.21.15.159:0/2730835935 wait complete. 2024-03-19T01:44:25.845 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 1 2024-03-19T01:44:26.629 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 podman[67090]: 2024-03-19 01:44:26.510577535 +0000 UTC m=+0.018680164 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:44:26.629 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 podman[67090]: 2024-03-19 01:44:26.629344595 +0000 UTC m=+0.137447221 container create 31e125f9f5c4c78d9bae217fe93b48b9a426a5f2b18374da8176f39ece1a8074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-19T01:44:26.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:26 smithi092 ceph-mon[62574]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:26 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/2730835935' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:44:26.890 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 podman[67090]: 2024-03-19 01:44:26.839936139 +0000 UTC m=+0.348038799 container init 31e125f9f5c4c78d9bae217fe93b48b9a426a5f2b18374da8176f39ece1a8074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 podman[67090]: 2024-03-19 01:44:26.842940906 +0000 UTC m=+0.351043522 container start 31e125f9f5c4c78d9bae217fe93b48b9a426a5f2b18374da8176f39ece1a8074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: set uid:gid to 167:167 (ceph:ceph) 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: ceph version 19.0.0-2206-gb6ae2491 (b6ae24918b03b1b8a19c9263a50655bba80bb810) squid (dev), process ceph-mon, pid 2 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pidfile_write: ignore empty --pid-file 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: load: jerasure load: lrc 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: RocksDB version: 7.9.2 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Git sha 0 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Compile date 2024-03-18 18:57:12 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: DB SUMMARY 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: DB Session ID: 3ZR94YYYC2WH5TD4HW49 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: CURRENT file: CURRENT 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: IDENTITY file: IDENTITY 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-19T01:44:26.891 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.error_if_exists: 0 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.create_if_missing: 0 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.paranoid_checks: 1 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.env: 0x5569971fbc60 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.fs: PosixFileSystem 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.info_log: 0x5569990b3ae0 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_file_opening_threads: 16 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.statistics: (nil) 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.use_fsync: 0 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_log_file_size: 0 2024-03-19T01:44:26.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.keep_log_file_num: 1000 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.recycle_log_file_num: 0 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.allow_fallocate: 1 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.allow_mmap_reads: 0 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.allow_mmap_writes: 0 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.use_direct_reads: 0 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.create_missing_column_families: 0 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.db_log_dir: 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.wal_dir: 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-19T01:44:26.893 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.advise_random_on_open: 1 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.db_write_buffer_size: 0 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.write_buffer_manager: 0x5569990861e0 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.rate_limiter: (nil) 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.wal_recovery_mode: 2 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.enable_thread_tracking: 0 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.enable_pipelined_write: 0 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.unordered_write: 0 2024-03-19T01:44:26.894 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-19T01:44:26.895 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-19T01:44:26.895 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-19T01:44:26.895 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-19T01:44:26.895 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.row_cache: None 2024-03-19T01:44:26.895 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.wal_filter: None 2024-03-19T01:44:26.896 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.allow_ingest_behind: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.two_write_queues: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.manual_wal_flush: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.wal_compression: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.atomic_flush: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.log_readahead_size: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.best_efforts_recovery: 0 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-19T01:44:26.897 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.allow_data_in_errors: 0 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.db_host_id: __hostname__ 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_background_jobs: 2 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_background_compactions: -1 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_subcompactions: 1 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_total_wal_size: 0 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_open_files: -1 2024-03-19T01:44:26.898 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bytes_per_sync: 0 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_readahead_size: 0 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_background_flushes: -1 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Compression algorithms supported: 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: kZSTD supported: 0 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: kXpressCompression supported: 0 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: kLZ4HCCompression supported: 1 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: kZlibCompression supported: 1 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: kSnappyCompression supported: 1 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: kLZ4Compression supported: 1 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: kBZip2Compression supported: 0 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-19T01:44:26.899 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.merge_operator: 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_filter: None 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_filter_factory: None 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.sst_partitioner_factory: None 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5569990b3be0) 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout: cache_index_and_filter_blocks: 1 2024-03-19T01:44:26.900 INFO:journalctl@ceph.mon.c.smithi159.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: pin_top_level_index_and_filter: 1 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: index_type: 0 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: data_block_index_type: 0 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: index_shortening: 1 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: checksum: 4 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: no_block_cache: 0 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache: 0x5569990b9090 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache_name: BinnedLRUCache 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache_options: 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: capacity : 536870912 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: num_shard_bits : 4 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: strict_capacity_limit : 0 2024-03-19T01:44:26.901 INFO:journalctl@ceph.mon.c.smithi159.stdout: high_pri_pool_ratio: 0.000 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_cache_compressed: (nil) 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: persistent_cache: (nil) 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_size: 4096 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_size_deviation: 10 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_restart_interval: 16 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: index_block_restart_interval: 1 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: metadata_block_size: 4096 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: partition_filters: 0 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: use_delta_encoding: 1 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: filter_policy: bloomfilter 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: whole_key_filtering: 1 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: verify_compression: 0 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: read_amp_bytes_per_bit: 0 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: format_version: 5 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: enable_index_compression: 1 2024-03-19T01:44:26.902 INFO:journalctl@ceph.mon.c.smithi159.stdout: block_align: 0 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout: max_auto_readahead_size: 262144 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout: prepopulate_block_cache: 0 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout: initial_auto_readahead_size: 8192 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout: num_file_reads_for_auto_readahead: 2 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.write_buffer_size: 33554432 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_write_buffer_number: 2 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression: NoCompression 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression: Disabled 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.prefix_extractor: nullptr 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.num_levels: 7 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-19T01:44:26.903 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression_opts.level: 32767 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression_opts.strategy: 0 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-19T01:44:26.904 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression_opts.enabled: false 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.target_file_size_base: 67108864 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-19T01:44:26.905 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.arena_block_size: 1048576 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.disable_auto_compactions: 0 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-19T01:44:26.906 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.table_properties_collectors: 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.inplace_update_support: 0 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-19T01:44:26.907 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.bloom_locality: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.max_successive_merges: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.paranoid_file_checks: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.force_consistency_checks: 1 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.report_bg_io_stats: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.ttl: 2592000 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.enable_blob_files: false 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.min_blob_size: 0 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.blob_file_size: 268435456 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-19T01:44:26.908 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.blob_file_starting_level: 0 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7dd82eb9-4277-41a9-ae32-0ba6bd3a063d 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710812666872932, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710812666873420, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710812666, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7dd82eb9-4277-41a9-ae32-0ba6bd3a063d", "db_session_id": "3ZR94YYYC2WH5TD4HW49", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710812666873501, "job": 1, "event": "recovery_finished"} 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-19T01:44:26.909 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5569991ac000 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: DB pointer 0x556999196000 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** DB Stats ** 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** Compaction Stats [default] ** 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T01:44:26.910 INFO:journalctl@ceph.mon.c.smithi159.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** Compaction Stats [default] ** 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-19T01:44:26.911 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout: Block cache BinnedLRUCache@0x5569990b9090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: using public_addr v2:172.21.15.159:0/0 -> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: starting mon.c rank -1 at public addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] at bind addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(???) e0 preinit fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).mds e1 new map 2024-03-19T01:44:26.912 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout: e1 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout: legacy client fscid: -1 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout: 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout: No filesystems configured 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-19T01:44:26.913 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Added host smithi144 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.914 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Updating smithi144:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Deploying cephadm binary to smithi159 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Updating smithi144:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Added host smithi159 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1432780990' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.916 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Updating smithi159:/etc/ceph/ceph.conf 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Updating smithi159:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Updating smithi159:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1432780990' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi092:172.21.15.92=a;smithi144:172.21.15.144=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:26.917 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Saving service mon spec with placement smithi092:172.21.15.92=a;smithi144:172.21.15.144=b;smithi159:172.21.15.159=c;count:3 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: Deploying daemon mon.c on smithi159 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/2730835935' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:44:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 ceph-mon[67110]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-19T01:44:27.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 bash[67090]: 31e125f9f5c4c78d9bae217fe93b48b9a426a5f2b18374da8176f39ece1a8074 2024-03-19T01:44:27.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:26 smithi159 systemd[1]: Started Ceph mon.c for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:44:27.406 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-19T01:44:27.406 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-19T01:44:27.640 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:44:32.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.225+0000 7fb53952d700 1 -- 172.21.15.159:0/3799881671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c0035c0 msgr2=0x7fb51c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:32.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.225+0000 7fb53952d700 1 --2- 172.21.15.159:0/3799881671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c0035c0 0x7fb51c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb534101d10 tx=0x7fb52400feb0 comp rx=0 tx=0).stop 2024-03-19T01:44:32.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.225+0000 7fb53952d700 1 -- 172.21.15.159:0/3799881671 shutdown_connections 2024-03-19T01:44:32.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.225+0000 7fb53952d700 1 --2- 172.21.15.159:0/3799881671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51c0035c0 0x7fb51c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:32.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.225+0000 7fb53952d700 1 -- 172.21.15.159:0/3799881671 >> 172.21.15.159:0/3799881671 conn(0x7fb5340fd8b0 msgr2=0x7fb5340ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:32.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.225+0000 7fb53952d700 1 -- 172.21.15.159:0/3799881671 shutdown_connections 2024-03-19T01:44:32.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.225+0000 7fb53952d700 1 -- 172.21.15.159:0/3799881671 wait complete. 2024-03-19T01:44:32.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.226+0000 7fb53952d700 1 Processor -- start 2024-03-19T01:44:32.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.226+0000 7fb53952d700 1 -- start start 2024-03-19T01:44:32.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.226+0000 7fb53952d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 0x7fb53419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:32.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.226+0000 7fb53952d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53419db80 0x7fb5341977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:32.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.226+0000 7fb53952d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb534109e70 con 0x7fb53419db80 2024-03-19T01:44:32.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.226+0000 7fb53952d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb534109cf0 con 0x7fb51c0035c0 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.226+0000 7fb532ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 0x7fb53419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb532ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 0x7fb53419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48022/0 (socket says 172.21.15.159:48022) 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb532ffd700 1 -- 172.21.15.159:0/63821063 learned_addr learned my addr 172.21.15.159:0/63821063 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb532ffd700 1 -- 172.21.15.159:0/63821063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 msgr2=0x7fb53419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb532ffd700 1 -- 172.21.15.159:0/63821063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 msgr2=0x7fb53419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb532ffd700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 0x7fb53419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb5327fc700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53419db80 0x7fb5341977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb532ffd700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 0x7fb53419d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb5327fc700 1 -- 172.21.15.159:0/63821063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 msgr2=0x7fb53419d640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb5327fc700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 0x7fb53419d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:32.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb5327fc700 1 -- 172.21.15.159:0/63821063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53419e1c0 con 0x7fb53419db80 2024-03-19T01:44:32.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb5327fc700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53419db80 0x7fb5341977c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb52800b4d0 tx=0x7fb52800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:32.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb51bfff700 1 -- 172.21.15.159:0/63821063 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb528011630 con 0x7fb53419db80 2024-03-19T01:44:32.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb51bfff700 1 -- 172.21.15.159:0/63821063 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb528011cb0 con 0x7fb53419db80 2024-03-19T01:44:32.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.227+0000 7fb51bfff700 1 -- 172.21.15.159:0/63821063 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb528010e60 con 0x7fb53419db80 2024-03-19T01:44:32.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.228+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb534197d60 con 0x7fb53419db80 2024-03-19T01:44:32.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.228+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5341982c0 con 0x7fb53419db80 2024-03-19T01:44:32.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.228+0000 7fb51bfff700 1 -- 172.21.15.159:0/63821063 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fb528018020 con 0x7fb53419db80 2024-03-19T01:44:32.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.229+0000 7fb51bfff700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb520038d10 0x7fb52003b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:32.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.229+0000 7fb51bfff700 1 -- 172.21.15.159:0/63821063 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb52804baa0 con 0x7fb53419db80 2024-03-19T01:44:32.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.229+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb534103520 con 0x7fb53419db80 2024-03-19T01:44:32.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.229+0000 7fb532ffd700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb520038d10 0x7fb52003b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:32.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.230+0000 7fb532ffd700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb520038d10 0x7fb52003b1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb52400fa60 tx=0x7fb524005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:32.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.235+0000 7fb51bfff700 1 -- 172.21.15.159:0/63821063 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb52801f020 con 0x7fb53419db80 2024-03-19T01:44:32.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:44:32.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: mon.a calling monitor election 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: mon.c calling monitor election 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: monmap epoch 2 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: last_changed 2024-03-19T01:44:26.895788+0000 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: created 2024-03-19T01:42:32.859371+0000 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: min_mon_release 19 (squid) 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: election_strategy: 1 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: fsmap 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: mgrmap e13: x(active, since 52s) 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: overall HEALTH_OK 2024-03-19T01:44:32.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:32.313 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:32.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: mon.a calling monitor election 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: mon.c calling monitor election 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:32.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: monmap epoch 2 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: last_changed 2024-03-19T01:44:26.895788+0000 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: created 2024-03-19T01:42:32.859371+0000 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: min_mon_release 19 (squid) 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: election_strategy: 1 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: fsmap 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: mgrmap e13: x(active, since 52s) 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: overall HEALTH_OK 2024-03-19T01:44:32.457 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:32.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:32.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.512+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb534198630 con 0x7fb53419db80 2024-03-19T01:44:32.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.513+0000 7fb51bfff700 1 -- 172.21.15.159:0/63821063 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fb528018310 con 0x7fb53419db80 2024-03-19T01:44:32.514 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:44:32.514 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":2,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","modified":"2024-03-19T01:44:26.895788Z","created":"2024-03-19T01:42:32.859371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.515+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb520038d10 msgr2=0x7fb52003b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.515+0000 7fb53952d700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb520038d10 0x7fb52003b1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb52400fa60 tx=0x7fb524005a90 comp rx=0 tx=0).stop 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.515+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53419db80 msgr2=0x7fb5341977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.515+0000 7fb53952d700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53419db80 0x7fb5341977c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb52800b4d0 tx=0x7fb52800b890 comp rx=0 tx=0).stop 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.515+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 shutdown_connections 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.515+0000 7fb53952d700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb520038d10 0x7fb52003b1d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.515+0000 7fb53952d700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb51c0035c0 0x7fb53419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.515+0000 7fb53952d700 1 --2- 172.21.15.159:0/63821063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb53419db80 0x7fb5341977c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.515+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 >> 172.21.15.159:0/63821063 conn(0x7fb5340fd8b0 msgr2=0x7fb5340feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.516+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 shutdown_connections 2024-03-19T01:44:32.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:32.516+0000 7fb53952d700 1 -- 172.21.15.159:0/63821063 wait complete. 2024-03-19T01:44:32.517 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 2 2024-03-19T01:44:33.110 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:33.111 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:33.111 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:33.111 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:33 smithi159 ceph-mon[67110]: Deploying daemon mon.b on smithi144 2024-03-19T01:44:33.111 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:33 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/63821063' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:44:33.111 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:33.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:33.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:33.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:33.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:33 smithi092 ceph-mon[62574]: Deploying daemon mon.b on smithi144 2024-03-19T01:44:33.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:33 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/63821063' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-19T01:44:33.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:34.138 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-19T01:44:34.138 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-19T01:44:34.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:34 smithi092 ceph-mon[62574]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:34.384 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:44:34.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:34 smithi159 ceph-mon[67110]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:36.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:35 smithi144 systemd[1]: Started Ceph mon.b for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:44:41.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: mon.c calling monitor election 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: mon.a calling monitor election 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:44:41.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: monmap epoch 3 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: last_changed 2024-03-19T01:44:35.834200+0000 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: created 2024-03-19T01:42:32.859371+0000 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: min_mon_release 19 (squid) 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: election_strategy: 1 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: fsmap 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: mgrmap e13: x(active, since 61s) 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-19T01:44:41.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-19T01:44:41.209 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: mon.b (rank 2) addr [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] is down (out of quorum) 2024-03-19T01:44:41.209 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:41.209 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:41.209 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:41.209 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:41.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: mon.c calling monitor election 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: mon.a calling monitor election 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: monmap epoch 3 2024-03-19T01:44:41.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: last_changed 2024-03-19T01:44:35.834200+0000 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: created 2024-03-19T01:42:32.859371+0000 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: min_mon_release 19 (squid) 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: election_strategy: 1 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: fsmap 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: mgrmap e13: x(active, since 61s) 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-19T01:44:41.312 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: mon.b (rank 2) addr [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] is down (out of quorum) 2024-03-19T01:44:41.313 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:41.313 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:41.313 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:41.313 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:41.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.839+0000 7ff23fae2700 1 -- 172.21.15.159:0/760944301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff224003a40 msgr2=0x7ff224003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:41.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.839+0000 7ff23fae2700 1 --2- 172.21.15.159:0/760944301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff224003a40 0x7ff224003e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff238101da0 tx=0x7ff23400fe90 comp rx=0 tx=0).stop 2024-03-19T01:44:41.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.839+0000 7ff23fae2700 1 -- 172.21.15.159:0/760944301 shutdown_connections 2024-03-19T01:44:41.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.839+0000 7ff23fae2700 1 --2- 172.21.15.159:0/760944301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff224003a40 0x7ff224003e20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:41.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.839+0000 7ff23fae2700 1 -- 172.21.15.159:0/760944301 >> 172.21.15.159:0/760944301 conn(0x7ff2380fc7b0 msgr2=0x7ff2380febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:41.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.839+0000 7ff23fae2700 1 -- 172.21.15.159:0/760944301 shutdown_connections 2024-03-19T01:44:41.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.839+0000 7ff23fae2700 1 -- 172.21.15.159:0/760944301 wait complete. 2024-03-19T01:44:41.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.840+0000 7ff23fae2700 1 Processor -- start 2024-03-19T01:44:41.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.840+0000 7ff23fae2700 1 -- start start 2024-03-19T01:44:41.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.840+0000 7ff23fae2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff224003a40 0x7ff238198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:41.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.840+0000 7ff23fae2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 0x7ff238199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:41.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.840+0000 7ff23fae2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2381a19f0 0x7ff2381a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.840+0000 7ff23fae2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff238108d00 con 0x7ff224003a40 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.840+0000 7ff23fae2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff238108b80 con 0x7ff2381a19f0 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.840+0000 7ff23fae2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff238108e80 con 0x7ff238198ec0 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d07d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 0x7ff238199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d07d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 0x7ff238199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46790/0 (socket says 172.21.15.159:46790) 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23e07f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2381a19f0 0x7ff2381a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d07d700 1 -- 172.21.15.159:0/4190927597 learned_addr learned my addr 172.21.15.159:0/4190927597 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d87e700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff224003a40 0x7ff238198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23e07f700 1 -- 172.21.15.159:0/4190927597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2381a19f0 msgr2=0x7ff2381a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23e07f700 1 -- 172.21.15.159:0/4190927597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2381a19f0 msgr2=0x7ff2381a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23e07f700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2381a19f0 0x7ff2381a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d07d700 1 -- 172.21.15.159:0/4190927597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 msgr2=0x7ff238199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-19T01:44:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23e07f700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2381a19f0 0x7ff2381a3de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d07d700 1 -- 172.21.15.159:0/4190927597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 msgr2=0x7ff238199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d07d700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 0x7ff238199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d07d700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 0x7ff238199340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d87e700 1 -- 172.21.15.159:0/4190927597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 msgr2=0x7ff238199340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d87e700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 0x7ff238199340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d87e700 1 -- 172.21.15.159:0/4190927597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2381a19f0 msgr2=0x7ff2381a3de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d87e700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2381a19f0 0x7ff2381a3de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d87e700 1 -- 172.21.15.159:0/4190927597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2381a4620 con 0x7ff224003a40 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23d87e700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff224003a40 0x7ff238198980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff238101da0 tx=0x7ff234012cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff22effd700 1 -- 172.21.15.159:0/4190927597 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234011de0 con 0x7ff224003a40 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2381a48b0 con 0x7ff224003a40 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.841+0000 7ff22effd700 1 -- 172.21.15.159:0/4190927597 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23402a070 con 0x7ff224003a40 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.842+0000 7ff22effd700 1 -- 172.21.15.159:0/4190927597 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23401dac0 con 0x7ff224003a40 2024-03-19T01:44:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.842+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2381a4de0 con 0x7ff224003a40 2024-03-19T01:44:41.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.842+0000 7ff22effd700 1 -- 172.21.15.159:0/4190927597 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7ff23402c020 con 0x7ff224003a40 2024-03-19T01:44:41.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.842+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff238063940 con 0x7ff224003a40 2024-03-19T01:44:41.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.843+0000 7ff22effd700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff21c039450 0x7ff21c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:41.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.843+0000 7ff22effd700 1 -- 172.21.15.159:0/4190927597 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff23404e770 con 0x7ff224003a40 2024-03-19T01:44:41.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.843+0000 7ff23d07d700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff21c039450 0x7ff21c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:41.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.843+0000 7ff23d07d700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff21c039450 0x7ff21c03b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff228009180 tx=0x7ff228008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:41.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:41.848+0000 7ff22effd700 1 -- 172.21.15.159:0/4190927597 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff2340263f0 con 0x7ff224003a40 2024-03-19T01:44:42.138 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:42.139 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:41 smithi159 ceph-mon[67110]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:42.139 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:42.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.153+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff23802dbb0 con 0x7ff224003a40 2024-03-19T01:44:42.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.154+0000 7ff22effd700 1 -- 172.21.15.159:0/4190927597 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7ff234003910 con 0x7ff224003a40 2024-03-19T01:44:42.154 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-19T01:44:42.154 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":3,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","modified":"2024-03-19T01:44:35.834200Z","created":"2024-03-19T01:42:32.859371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff21c039450 msgr2=0x7ff21c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff21c039450 0x7ff21c03b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff228009180 tx=0x7ff228008040 comp rx=0 tx=0).stop 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff224003a40 msgr2=0x7ff238198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff224003a40 0x7ff238198980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff238101da0 tx=0x7ff234012cb0 comp rx=0 tx=0).stop 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 shutdown_connections 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff21c039450 0x7ff21c03b910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2381a19f0 0x7ff2381a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff224003a40 0x7ff238198980 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 --2- 172.21.15.159:0/4190927597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff238198ec0 0x7ff238199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 >> 172.21.15.159:0/4190927597 conn(0x7ff2380fc7b0 msgr2=0x7ff2380fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 shutdown_connections 2024-03-19T01:44:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:42.156+0000 7ff23fae2700 1 -- 172.21.15.159:0/4190927597 wait complete. 2024-03-19T01:44:42.158 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 3 2024-03-19T01:44:42.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:42.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:41 smithi092 ceph-mon[62574]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:42.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:42.994 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-19T01:44:42.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-19T01:44:43.244 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:44:43.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: mon.c calling monitor election 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: mon.a calling monitor election 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: monmap epoch 3 2024-03-19T01:44:43.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: last_changed 2024-03-19T01:44:35.834200+0000 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: created 2024-03-19T01:42:32.859371+0000 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: min_mon_release 19 (squid) 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: election_strategy: 1 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: fsmap 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: mgrmap e13: x(active, since 61s) 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: mon.b (rank 2) addr [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] is down (out of quorum) 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:43.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:43.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:43.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: mon.b calling monitor election 2024-03-19T01:44:43.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: mon.b calling monitor election 2024-03-19T01:44:43.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: mon.c calling monitor election 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: mon.a calling monitor election 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: monmap epoch 3 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: last_changed 2024-03-19T01:44:35.834200+0000 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: created 2024-03-19T01:42:32.859371+0000 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: min_mon_release 19 (squid) 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: election_strategy: 1 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: fsmap 2024-03-19T01:44:43.670 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: mgrmap e13: x(active, since 63s) 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: Cluster is now healthy 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: overall HEALTH_OK 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.671 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:43.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:44:43.672 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:43.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: mon.b calling monitor election 2024-03-19T01:44:43.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: mon.b calling monitor election 2024-03-19T01:44:43.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: mon.c calling monitor election 2024-03-19T01:44:43.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: mon.a calling monitor election 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: monmap epoch 3 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: last_changed 2024-03-19T01:44:35.834200+0000 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: created 2024-03-19T01:42:32.859371+0000 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: min_mon_release 19 (squid) 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: election_strategy: 1 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: fsmap 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: mgrmap e13: x(active, since 63s) 2024-03-19T01:44:43.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: Cluster is now healthy 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: overall HEALTH_OK 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:43.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:44:43.708 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:43.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: mon.b calling monitor election 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: mon.b calling monitor election 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: mon.c calling monitor election 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: mon.a calling monitor election 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: monmap epoch 3 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: last_changed 2024-03-19T01:44:35.834200+0000 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: created 2024-03-19T01:42:32.859371+0000 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: min_mon_release 19 (squid) 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: election_strategy: 1 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: 0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-03-19T01:44:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: 2: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: fsmap 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: osdmap e4: 0 total, 0 up, 0 in 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: mgrmap e13: x(active, since 63s) 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: Cluster is now healthy 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: overall HEALTH_OK 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.849 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.849 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:43.849 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:43.849 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:44:43.849 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:44.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:44 smithi092 ceph-mon[62574]: Updating smithi092:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:44.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:44 smithi092 ceph-mon[62574]: Updating smithi144:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:44.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:44 smithi092 ceph-mon[62574]: Updating smithi159:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:44.542 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:44 smithi092 ceph-mon[62574]: Reconfiguring mon.a (unknown last config time)... 2024-03-19T01:44:44.542 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:44 smithi092 ceph-mon[62574]: Reconfiguring daemon mon.a on smithi092 2024-03-19T01:44:44.542 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:44 smithi092 ceph-mon[62574]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:44.542 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:44.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:44 smithi159 ceph-mon[67110]: Updating smithi092:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:44.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:44 smithi159 ceph-mon[67110]: Updating smithi144:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:44.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:44 smithi159 ceph-mon[67110]: Updating smithi159:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:44.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:44 smithi159 ceph-mon[67110]: Reconfiguring mon.a (unknown last config time)... 2024-03-19T01:44:44.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:44 smithi159 ceph-mon[67110]: Reconfiguring daemon mon.a on smithi092 2024-03-19T01:44:44.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:44 smithi159 ceph-mon[67110]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:44.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:44.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.741+0000 7f9176b94700 1 -- 172.21.15.92:0/4198602249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170101f50 msgr2=0x7f9170102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:44.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.741+0000 7f9176b94700 1 --2- 172.21.15.92:0/4198602249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170101f50 0x7f9170102330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f916000b0a0 tx=0x7f916000b3b0 comp rx=0 tx=0).stop 2024-03-19T01:44:44.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.741+0000 7f9176b94700 1 -- 172.21.15.92:0/4198602249 shutdown_connections 2024-03-19T01:44:44.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.741+0000 7f9176b94700 1 --2- 172.21.15.92:0/4198602249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170101f50 0x7f9170102330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:44.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.741+0000 7f9176b94700 1 -- 172.21.15.92:0/4198602249 >> 172.21.15.92:0/4198602249 conn(0x7f91700fd800 msgr2=0x7f91700ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.741+0000 7f9176b94700 1 -- 172.21.15.92:0/4198602249 shutdown_connections 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.741+0000 7f9176b94700 1 -- 172.21.15.92:0/4198602249 wait complete. 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9176b94700 1 Processor -- start 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9176b94700 1 -- start start 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9176b94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9170101f50 0x7f917019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9176b94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917019dc30 0x7f9170197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9176b94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170197e90 0x7f9170198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9176b94700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f917010c2e0 con 0x7f9170197e90 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9176b94700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f917010c160 con 0x7f917019dc30 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9176b94700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f917010c460 con 0x7f9170101f50 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9175131700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170197e90 0x7f9170198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9175131700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170197e90 0x7f9170198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53538/0 (socket says 172.21.15.92:53538) 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9175131700 1 -- 172.21.15.92:0/1793575915 learned_addr learned my addr 172.21.15.92:0/1793575915 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:44:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9174930700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9170101f50 0x7f917019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f916ffff700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917019dc30 0x7f9170197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9175131700 1 -- 172.21.15.92:0/1793575915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9170101f50 msgr2=0x7f917019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9175131700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9170101f50 0x7f917019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9175131700 1 -- 172.21.15.92:0/1793575915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917019dc30 msgr2=0x7f9170197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9175131700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917019dc30 0x7f9170197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.742+0000 7f9175131700 1 -- 172.21.15.92:0/1793575915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91701a45c0 con 0x7f9170197e90 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.743+0000 7f9175131700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170197e90 0x7f9170198340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f916800b550 tx=0x7f916800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.743+0000 7f916dffb700 1 -- 172.21.15.92:0/1793575915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168015020 con 0x7f9170197e90 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.743+0000 7f916dffb700 1 -- 172.21.15.92:0/1793575915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9168011d10 con 0x7f9170197e90 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.743+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91701a47f0 con 0x7f9170197e90 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.743+0000 7f916dffb700 1 -- 172.21.15.92:0/1793575915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916800f2e0 con 0x7f9170197e90 2024-03-19T01:44:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.743+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91701a4d30 con 0x7f9170197e90 2024-03-19T01:44:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.744+0000 7f916dffb700 1 -- 172.21.15.92:0/1793575915 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f9168018020 con 0x7f9170197e90 2024-03-19T01:44:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.744+0000 7f916dffb700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f915c039450 0x7f915c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.744+0000 7f916dffb700 1 -- 172.21.15.92:0/1793575915 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f916804b9c0 con 0x7f9170197e90 2024-03-19T01:44:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.744+0000 7f9174930700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f915c039450 0x7f915c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.744+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9170063940 con 0x7f9170197e90 2024-03-19T01:44:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.744+0000 7f9174930700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f915c039450 0x7f915c03b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f917010a1f0 tx=0x7f9160005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:44.749 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.749+0000 7f916dffb700 1 -- 172.21.15.92:0/1793575915 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f916801f020 con 0x7f9170197e90 2024-03-19T01:44:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:44 smithi144 ceph-mon[67199]: Updating smithi092:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:44 smithi144 ceph-mon[67199]: Updating smithi144:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:44 smithi144 ceph-mon[67199]: Updating smithi159:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-19T01:44:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:44 smithi144 ceph-mon[67199]: Reconfiguring mon.a (unknown last config time)... 2024-03-19T01:44:44.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:44 smithi144 ceph-mon[67199]: Reconfiguring daemon mon.a on smithi092 2024-03-19T01:44:44.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:44 smithi144 ceph-mon[67199]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:44.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:44:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.956+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f91701a4ff0 con 0x7f9170197e90 2024-03-19T01:44:44.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.957+0000 7f916dffb700 1 -- 172.21.15.92:0/1793575915 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f9168018310 con 0x7f9170197e90 2024-03-19T01:44:44.957 INFO:teuthology.orchestra.run.smithi092.stdout:# minimal ceph.conf for d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:44.957 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-03-19T01:44:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:44:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: mon_host = [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2024-03-19T01:44:44.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f915c039450 msgr2=0x7f915c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:44.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f915c039450 0x7f915c03b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f917010a1f0 tx=0x7f9160005040 comp rx=0 tx=0).stop 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170197e90 msgr2=0x7f9170198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170197e90 0x7f9170198340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f916800b550 tx=0x7f916800b910 comp rx=0 tx=0).stop 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 shutdown_connections 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f915c039450 0x7f915c03b910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9170197e90 0x7f9170198340 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9170101f50 0x7f917019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 --2- 172.21.15.92:0/1793575915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f917019dc30 0x7f9170197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 >> 172.21.15.92:0/1793575915 conn(0x7f91700fd800 msgr2=0x7f9170103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 shutdown_connections 2024-03-19T01:44:44.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:44.959+0000 7f9176b94700 1 -- 172.21.15.92:0/1793575915 wait complete. 2024-03-19T01:44:45.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:45.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:45 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1793575915' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:45.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:45.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:45 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1793575915' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:45.774 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:45.774 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:45 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1793575915' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:46.092 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-19T01:44:46.092 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:44:46.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-03-19T01:44:46.123 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:44:46.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:44:46.191 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:44:46.191 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-03-19T01:44:46.226 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:44:46.226 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:44:46.297 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:44:46.297 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2024-03-19T01:44:46.331 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:44:46.331 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-19T01:44:46.402 INFO:tasks.cephadm:Adding mgr.x on smithi092 2024-03-19T01:44:46.402 INFO:tasks.cephadm:Adding mgr.y on smithi144 2024-03-19T01:44:46.402 INFO:tasks.cephadm:Adding mgr.z on smithi159 2024-03-19T01:44:46.402 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi092=x;smithi144=y;smithi159=z' 2024-03-19T01:44:46.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:46 smithi144 ceph-mon[67199]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:46.679 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:44:46.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:46 smithi159 ceph-mon[67110]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:46 smithi092 ceph-mon[62574]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:47.726 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:47.726 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:47.726 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:47.726 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:44:47.726 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:44:47.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:48.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:48.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:48.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:48.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:44:48.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:48.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.231+0000 7fe378766700 1 -- 172.21.15.159:0/1212796952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3701034d0 msgr2=0x7fe3701038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:48.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.231+0000 7fe378766700 1 --2- 172.21.15.159:0/1212796952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3701034d0 0x7fe3701038f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009910 tx=0x7fe36c009c20 comp rx=0 tx=0).stop 2024-03-19T01:44:48.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.231+0000 7fe378766700 1 -- 172.21.15.159:0/1212796952 shutdown_connections 2024-03-19T01:44:48.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.231+0000 7fe378766700 1 --2- 172.21.15.159:0/1212796952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3701034d0 0x7fe3701038f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:48.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.231+0000 7fe378766700 1 -- 172.21.15.159:0/1212796952 >> 172.21.15.159:0/1212796952 conn(0x7fe3700fd430 msgr2=0x7fe3700ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:48.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.231+0000 7fe378766700 1 -- 172.21.15.159:0/1212796952 shutdown_connections 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.231+0000 7fe378766700 1 -- 172.21.15.159:0/1212796952 wait complete. 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.231+0000 7fe378766700 1 Processor -- start 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe378766700 1 -- start start 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe378766700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3701034d0 0x7fe3701977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe378766700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe370197d00 0x7fe37019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe378766700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe370198180 0x7fe37019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe378766700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe370107f40 con 0x7fe370197d00 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe378766700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe370107dc0 con 0x7fe3701034d0 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe378766700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3701080c0 con 0x7fe370198180 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe376502700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3701034d0 0x7fe3701977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe376502700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3701034d0 0x7fe3701977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.159:50426/0 (socket says 172.21.15.159:50426) 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe375d01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe370197d00 0x7fe37019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe376502700 1 -- 172.21.15.159:0/1922186073 learned_addr learned my addr 172.21.15.159:0/1922186073 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:44:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe376d03700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe370198180 0x7fe37019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:48.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe375d01700 1 -- 172.21.15.159:0/1922186073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe370198180 msgr2=0x7fe37019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:48.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe375d01700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe370198180 0x7fe37019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:48.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe375d01700 1 -- 172.21.15.159:0/1922186073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3701034d0 msgr2=0x7fe3701977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:48.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe375d01700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3701034d0 0x7fe3701977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:48.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe375d01700 1 -- 172.21.15.159:0/1922186073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37019fb80 con 0x7fe370197d00 2024-03-19T01:44:48.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.232+0000 7fe376502700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3701034d0 0x7fe3701977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:44:48.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.233+0000 7fe375d01700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe370197d00 0x7fe37019cd20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b550 tx=0x7fe36000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:48.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.233+0000 7fe3677fe700 1 -- 172.21.15.159:0/1922186073 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360004230 con 0x7fe370197d00 2024-03-19T01:44:48.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.233+0000 7fe3677fe700 1 -- 172.21.15.159:0/1922186073 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3600043d0 con 0x7fe370197d00 2024-03-19T01:44:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.233+0000 7fe3677fe700 1 -- 172.21.15.159:0/1922186073 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360011b20 con 0x7fe370197d00 2024-03-19T01:44:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.233+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37019fe70 con 0x7fe370197d00 2024-03-19T01:44:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.233+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3701a03f0 con 0x7fe370197d00 2024-03-19T01:44:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.234+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe370063940 con 0x7fe370197d00 2024-03-19T01:44:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.234+0000 7fe3677fe700 1 -- 172.21.15.159:0/1922186073 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fe360011d00 con 0x7fe370197d00 2024-03-19T01:44:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.234+0000 7fe3677fe700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe35c039450 0x7fe35c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.234+0000 7fe3677fe700 1 -- 172.21.15.159:0/1922186073 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe36004baa0 con 0x7fe370197d00 2024-03-19T01:44:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.234+0000 7fe376502700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe35c039450 0x7fe35c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.235+0000 7fe376502700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe35c039450 0x7fe35c03b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe36c006010 tx=0x7fe36c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:48.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.239+0000 7fe3677fe700 1 -- 172.21.15.159:0/1922186073 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe3600204e0 con 0x7fe370197d00 2024-03-19T01:44:48.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.452+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi144=y;smithi159=z", "target": ["mon-mgr", ""]}) -- 0x7fe3700628a0 con 0x7fe35c039450 2024-03-19T01:44:48.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.460+0000 7fe3677fe700 1 -- 172.21.15.159:0/1922186073 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe3700628a0 con 0x7fe35c039450 2024-03-19T01:44:48.461 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mgr update... 2024-03-19T01:44:48.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe35c039450 msgr2=0x7fe35c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:48.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe35c039450 0x7fe35c03b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe36c006010 tx=0x7fe36c01a040 comp rx=0 tx=0).stop 2024-03-19T01:44:48.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe370197d00 msgr2=0x7fe37019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:48.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe370197d00 0x7fe37019cd20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b550 tx=0x7fe36000b860 comp rx=0 tx=0).stop 2024-03-19T01:44:48.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 shutdown_connections 2024-03-19T01:44:48.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe35c039450 0x7fe35c03b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:48.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe370198180 0x7fe37019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:48.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3701034d0 0x7fe3701977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:48.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 --2- 172.21.15.159:0/1922186073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe370197d00 0x7fe37019cd20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:48.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 >> 172.21.15.159:0/1922186073 conn(0x7fe3700fd430 msgr2=0x7fe3700fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:48.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 shutdown_connections 2024-03-19T01:44:48.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:44:48.463+0000 7fe378766700 1 -- 172.21.15.159:0/1922186073 wait complete. 2024-03-19T01:44:48.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:48 smithi159 ceph-mon[67110]: Reconfiguring mon.b (monmap changed)... 2024-03-19T01:44:48.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:48 smithi159 ceph-mon[67110]: Reconfiguring daemon mon.b on smithi144 2024-03-19T01:44:48.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:48 smithi159 ceph-mon[67110]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:48.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:48.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:48.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:48.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:44:48.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:48.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:48.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:48 smithi144 ceph-mon[67199]: Reconfiguring mon.b (monmap changed)... 2024-03-19T01:44:48.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:48 smithi144 ceph-mon[67199]: Reconfiguring daemon mon.b on smithi144 2024-03-19T01:44:48.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:48 smithi144 ceph-mon[67199]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:44:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:49.007 DEBUG:teuthology.orchestra.run.smithi144:mgr.y> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.y.service 2024-03-19T01:44:49.010 DEBUG:teuthology.orchestra.run.smithi159:mgr.z> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.z.service 2024-03-19T01:44:49.013 INFO:tasks.cephadm:Deploying OSDs... 2024-03-19T01:44:49.013 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:44:49.013 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-03-19T01:44:49.031 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-19T01:44:49.032 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-03-19T01:44:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:48 smithi092 ceph-mon[62574]: Reconfiguring mon.b (monmap changed)... 2024-03-19T01:44:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:48 smithi092 ceph-mon[62574]: Reconfiguring daemon mon.b on smithi144 2024-03-19T01:44:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:48 smithi092 ceph-mon[62574]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-19T01:44:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-19T01:44:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:49.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-19T01:44:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:49.077 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-19T01:44:49.077 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:49.077 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:49.077 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-19 01:43:52.620627955 +0000 2024-03-19T01:44:49.077 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-19 01:16:44.953870574 +0000 2024-03-19T01:44:49.077 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-19 01:16:44.953870574 +0000 2024-03-19T01:44:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-03-19 01:16:44.953870574 +0000 2024-03-19T01:44:49.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-19T01:44:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-19T01:44:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-19T01:44:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000229877 s, 2.2 MB/s 2024-03-19T01:44:49.149 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-19T01:44:49.206 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-03-19T01:44:49.263 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-19T01:44:49.263 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:49.263 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-19T01:44:49.263 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:49.263 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:49.263 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-19 01:43:52.653627008 +0000 2024-03-19T01:44:49.263 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-19 01:16:45.294860940 +0000 2024-03-19T01:44:49.263 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-19 01:16:45.294860940 +0000 2024-03-19T01:44:49.263 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-03-19 01:16:45.294860940 +0000 2024-03-19T01:44:49.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-19T01:44:49.330 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-19T01:44:49.330 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-19T01:44:49.330 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000244578 s, 2.1 MB/s 2024-03-19T01:44:49.332 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-19T01:44:49.389 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-03-19T01:44:49.445 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-19T01:44:49.445 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:49.445 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-19T01:44:49.445 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:49.445 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:49.445 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-19 01:43:52.685626089 +0000 2024-03-19T01:44:49.445 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-19 01:16:45.648850938 +0000 2024-03-19T01:44:49.445 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-19 01:16:45.648850938 +0000 2024-03-19T01:44:49.445 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-03-19 01:16:45.648850938 +0000 2024-03-19T01:44:49.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-19T01:44:49.511 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-19T01:44:49.511 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-19T01:44:49.511 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000212033 s, 2.4 MB/s 2024-03-19T01:44:49.513 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-19T01:44:49.570 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-03-19T01:44:49.627 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-19T01:44:49.627 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:49.627 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-19T01:44:49.627 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:49.627 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:49.627 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-19 01:43:52.717625171 +0000 2024-03-19T01:44:49.628 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-19 01:16:45.990841275 +0000 2024-03-19T01:44:49.628 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-19 01:16:45.990841275 +0000 2024-03-19T01:44:49.628 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-03-19 01:16:45.990841275 +0000 2024-03-19T01:44:49.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-19T01:44:49.694 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-19T01:44:49.694 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-19T01:44:49.694 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000222119 s, 2.3 MB/s 2024-03-19T01:44:49.696 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-19T01:44:49.753 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-19T01:44:49.753 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-03-19T01:44:49.772 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-19T01:44:49.773 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-03-19T01:44:49.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:49 smithi092 ceph-mon[62574]: Reconfiguring mon.c (monmap changed)... 2024-03-19T01:44:49.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:49 smithi092 ceph-mon[62574]: Reconfiguring daemon mon.c on smithi159 2024-03-19T01:44:49.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:49 smithi092 ceph-mon[62574]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi144=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:49.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:49 smithi092 ceph-mon[62574]: Saving service mgr spec with placement smithi092=x;smithi144=y;smithi159=z;count:3 2024-03-19T01:44:49.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:49 smithi092 ceph-mon[62574]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:49.831 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-19T01:44:49.831 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:49.831 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-19T01:44:49.831 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-19 01:44:35.280253742 +0000 2024-03-19T01:44:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-19 01:19:12.051394344 +0000 2024-03-19T01:44:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-19 01:19:12.051394344 +0000 2024-03-19T01:44:49.832 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-19 01:19:12.051394344 +0000 2024-03-19T01:44:49.832 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-19T01:44:49.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:49 smithi144 ceph-mon[67199]: Reconfiguring mon.c (monmap changed)... 2024-03-19T01:44:49.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:49 smithi144 ceph-mon[67199]: Reconfiguring daemon mon.c on smithi159 2024-03-19T01:44:49.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:49 smithi144 ceph-mon[67199]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi144=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:49.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:49 smithi144 ceph-mon[67199]: Saving service mgr spec with placement smithi092=x;smithi144=y;smithi159=z;count:3 2024-03-19T01:44:49.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:49 smithi144 ceph-mon[67199]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:49.876 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-19T01:44:49.876 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-19T01:44:49.876 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.0002667 s, 1.9 MB/s 2024-03-19T01:44:49.878 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-19T01:44:49.936 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-03-19T01:44:49.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:49 smithi159 ceph-mon[67110]: Reconfiguring mon.c (monmap changed)... 2024-03-19T01:44:49.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:49 smithi159 ceph-mon[67110]: Reconfiguring daemon mon.c on smithi159 2024-03-19T01:44:49.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:49 smithi159 ceph-mon[67110]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi092=x;smithi144=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:49.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:49 smithi159 ceph-mon[67110]: Saving service mgr spec with placement smithi092=x;smithi144=y;smithi159=z;count:3 2024-03-19T01:44:49.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:49 smithi159 ceph-mon[67110]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:49.993 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-19T01:44:49.993 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:49.993 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-19T01:44:49.993 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:49.993 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:49.993 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-19 01:44:35.280253742 +0000 2024-03-19T01:44:49.993 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-19 01:19:12.393388463 +0000 2024-03-19T01:44:49.993 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-19 01:19:12.393388463 +0000 2024-03-19T01:44:49.993 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-19 01:19:12.393388463 +0000 2024-03-19T01:44:49.994 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-19T01:44:50.061 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-19T01:44:50.061 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-19T01:44:50.061 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000187194 s, 2.7 MB/s 2024-03-19T01:44:50.062 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-19T01:44:50.119 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-03-19T01:44:50.175 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-19T01:44:50.175 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:50.175 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-19T01:44:50.175 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:50.175 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:50.175 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-19 01:44:35.280253742 +0000 2024-03-19T01:44:50.175 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-19 01:19:12.734382599 +0000 2024-03-19T01:44:50.176 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-19 01:19:12.734382599 +0000 2024-03-19T01:44:50.176 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-19 01:19:12.734382599 +0000 2024-03-19T01:44:50.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-19T01:44:50.242 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-19T01:44:50.242 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-19T01:44:50.243 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000197672 s, 2.6 MB/s 2024-03-19T01:44:50.244 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-19T01:44:50.300 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-03-19T01:44:50.355 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-19T01:44:50.355 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:50.356 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-03-19T01:44:50.356 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:50.356 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:50.356 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-19 01:44:35.281253725 +0000 2024-03-19T01:44:50.356 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-19 01:19:13.067376873 +0000 2024-03-19T01:44:50.356 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-19 01:19:13.067376873 +0000 2024-03-19T01:44:50.356 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-19 01:19:13.067376873 +0000 2024-03-19T01:44:50.356 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-19T01:44:50.422 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-19T01:44:50.422 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-19T01:44:50.423 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000206976 s, 2.5 MB/s 2024-03-19T01:44:50.424 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-19T01:44:50.480 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-19T01:44:50.480 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-03-19T01:44:50.496 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-19T01:44:50.496 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-03-19T01:44:50.551 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-19T01:44:50.551 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:50.551 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-03-19T01:44:50.551 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:50.551 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:50.551 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-19 01:44:25.907755871 +0000 2024-03-19T01:44:50.551 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-19 01:23:51.135360890 +0000 2024-03-19T01:44:50.551 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-19 01:23:51.135360890 +0000 2024-03-19T01:44:50.551 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-19 01:23:51.135360890 +0000 2024-03-19T01:44:50.551 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-19T01:44:50.618 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-19T01:44:50.618 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-19T01:44:50.618 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000252514 s, 2.0 MB/s 2024-03-19T01:44:50.619 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-19T01:44:50.675 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-03-19T01:44:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-19T01:44:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:50.731 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-19T01:44:50.731 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:50.731 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:50.731 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-19 01:44:25.907755871 +0000 2024-03-19T01:44:50.731 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-19 01:23:51.485354324 +0000 2024-03-19T01:44:50.731 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-19 01:23:51.485354324 +0000 2024-03-19T01:44:50.731 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-19 01:23:51.485354324 +0000 2024-03-19T01:44:50.731 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-19T01:44:50.797 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-19T01:44:50.797 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-19T01:44:50.797 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000210657 s, 2.4 MB/s 2024-03-19T01:44:50.799 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-19T01:44:50.854 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-03-19T01:44:50.910 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-19T01:44:50.910 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:50.911 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-19T01:44:50.911 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:50.911 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:50.911 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-19 01:44:25.907755871 +0000 2024-03-19T01:44:50.911 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-19 01:23:51.814348151 +0000 2024-03-19T01:44:50.911 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-19 01:23:51.814348151 +0000 2024-03-19T01:44:50.911 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-19 01:23:51.814348151 +0000 2024-03-19T01:44:50.911 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-19T01:44:50.976 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-19T01:44:50.977 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-19T01:44:50.977 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000220185 s, 2.3 MB/s 2024-03-19T01:44:50.978 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-19T01:44:51.034 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-03-19T01:44:51.089 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-19T01:44:51.089 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-19T01:44:51.089 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-19T01:44:51.090 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:44:51.090 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-19T01:44:51.090 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-19 01:44:25.908755852 +0000 2024-03-19T01:44:51.090 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-19 01:23:52.158341698 +0000 2024-03-19T01:44:51.090 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-19 01:23:52.158341698 +0000 2024-03-19T01:44:51.090 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-19 01:23:52.158341698 +0000 2024-03-19T01:44:51.090 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-19T01:44:51.154 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-19T01:44:51.154 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-19T01:44:51.154 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.00018942 s, 2.7 MB/s 2024-03-19T01:44:51.155 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-19T01:44:51.211 INFO:tasks.cephadm:Deploying osd.0 on smithi092 with /dev/vg_nvme/lv_4... 2024-03-19T01:44:51.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-19T01:44:51.403 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:51.403 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:51.403 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:51.403 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:51.403 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:51.403 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:51.404 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:44:51.404 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:44:51.404 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:44:51.404 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:51 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:44:51.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:51.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:51.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:51.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:51.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:51.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:51.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:44:51.498 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:44:51.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:44:51.499 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:51 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:51.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:51.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:51.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:51.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:51.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:51.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:51.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:44:51.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:44:51.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:44:51.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:51 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:52.362 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:52 smithi159 ceph-mon[67110]: Deploying daemon mgr.z on smithi159 2024-03-19T01:44:52.362 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:52 smithi159 ceph-mon[67110]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:52.511 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:52 smithi144 ceph-mon[67199]: Deploying daemon mgr.z on smithi159 2024-03-19T01:44:52.511 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:52 smithi144 ceph-mon[67199]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:52.523 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:52 smithi092 ceph-mon[62574]: Deploying daemon mgr.z on smithi159 2024-03-19T01:44:52.523 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:52 smithi092 ceph-mon[62574]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:52.955 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:52 smithi159 systemd[1]: Starting Ceph mgr.z for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:44:53.294 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:53 smithi159 podman[68963]: 2024-03-19 01:44:52.939907812 +0000 UTC m=+0.017525172 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:44:53.294 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:53 smithi159 podman[68963]: 2024-03-19 01:44:53.064895186 +0000 UTC m=+0.142512532 container create 259c300c508e5f0962f03c3202de3151e822ddde0f2f492032dd2074f84fc8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-19T01:44:53.294 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:53 smithi159 podman[68963]: 2024-03-19 01:44:53.267189714 +0000 UTC m=+0.344807084 container init 259c300c508e5f0962f03c3202de3151e822ddde0f2f492032dd2074f84fc8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-03-19T01:44:53.294 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:53 smithi159 podman[68963]: 2024-03-19 01:44:53.269633494 +0000 UTC m=+0.347250849 container start 259c300c508e5f0962f03c3202de3151e822ddde0f2f492032dd2074f84fc8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-19T01:44:53.294 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:53 smithi159 ceph-mgr[68982]: -- 172.21.15.159:0/3582340086 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56362242c5a0 con 0x563622400c00 2024-03-19T01:44:53.375 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-19T01:44:53.375 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-19T01:44:53.375 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-03-19T01:44:53.375 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-03-19T01:44:53.375 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114178 s, 918 MB/s 2024-03-19T01:44:53.375 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-03-19T01:44:53.705 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:53 smithi159 bash[68963]: 259c300c508e5f0962f03c3202de3151e822ddde0f2f492032dd2074f84fc8c4 2024-03-19T01:44:53.705 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:53 smithi159 systemd[1]: Started Ceph mgr.z for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:44:54.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-03-19T01:44:54.205 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:53.809+0000 7fd648e1a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:44:54.205 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:53.901+0000 7fd648e1a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:44:54.258 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:44:54.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:54 smithi092 ceph-mon[62574]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:54.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:54.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:54.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:54.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:44:54.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:44:54.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:44:54.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:54.591 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:54 smithi092 ceph-mon[62574]: Deploying daemon mgr.y on smithi144 2024-03-19T01:44:54.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:54 smithi144 ceph-mon[67199]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:54.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:54.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:54.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:54.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:44:54.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:44:54.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:44:54.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:54.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:54 smithi144 ceph-mon[67199]: Deploying daemon mgr.y on smithi144 2024-03-19T01:44:54.705 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:54.542+0000 7fd648e1a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:44:54.705 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:54.632+0000 7fd648e1a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:44:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-mon[67110]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:44:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-19T01:44:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:44:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-mon[67110]: Deploying daemon mgr.y on smithi144 2024-03-19T01:44:55.205 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:54.773+0000 7fd648e1a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:44:55.205 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:54.867+0000 7fd648e1a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:44:55.205 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:54.949+0000 7fd648e1a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:44:55.493 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:55.208+0000 7fd648e1a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:44:55.494 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:55.316+0000 7fd648e1a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:44:55.494 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:55.399+0000 7fd648e1a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:44:55.494 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:55.493+0000 7fd648e1a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:44:55.596 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:55 smithi144 systemd[1]: Starting Ceph mgr.y for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:44:55.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.774+0000 7fd292d94700 1 -- 172.21.15.92:0/2330853816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c103f10 msgr2=0x7fd28c10a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:55.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.774+0000 7fd292d94700 1 --2- 172.21.15.92:0/2330853816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c103f10 0x7fd28c10a7c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd28400b0d0 tx=0x7fd28400b3e0 comp rx=0 tx=0).stop 2024-03-19T01:44:55.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.774+0000 7fd292d94700 1 -- 172.21.15.92:0/2330853816 shutdown_connections 2024-03-19T01:44:55.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.774+0000 7fd292d94700 1 --2- 172.21.15.92:0/2330853816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c103f10 0x7fd28c10a7c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:55.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.774+0000 7fd292d94700 1 --2- 172.21.15.92:0/2330853816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c102370 0x7fd28c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:55.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.774+0000 7fd292d94700 1 --2- 172.21.15.92:0/2330853816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd28c103570 0x7fd28c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:55.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.774+0000 7fd292d94700 1 -- 172.21.15.92:0/2330853816 >> 172.21.15.92:0/2330853816 conn(0x7fd28c0fd910 msgr2=0x7fd28c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:44:55.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.774+0000 7fd292d94700 1 -- 172.21.15.92:0/2330853816 shutdown_connections 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.774+0000 7fd292d94700 1 -- 172.21.15.92:0/2330853816 wait complete. 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd292d94700 1 Processor -- start 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd292d94700 1 -- start start 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd292d94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c102370 0x7fd28c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd292d94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd28c103570 0x7fd28c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd292d94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c103f10 0x7fd28c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd292d94700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10e000 con 0x7fd28c102370 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd292d94700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10de80 con 0x7fd28c103f10 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd292d94700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10e180 con 0x7fd28c103570 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd290b30700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c102370 0x7fd28c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd290b30700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c102370 0x7fd28c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41260/0 (socket says 172.21.15.92:41260) 2024-03-19T01:44:55.778 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd290b30700 1 -- 172.21.15.92:0/2430309893 learned_addr learned my addr 172.21.15.92:0/2430309893 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd28bfff700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd28c103570 0x7fd28c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd291331700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c103f10 0x7fd28c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd290b30700 1 -- 172.21.15.92:0/2430309893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd28c103570 msgr2=0x7fd28c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd290b30700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd28c103570 0x7fd28c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd290b30700 1 -- 172.21.15.92:0/2430309893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c103f10 msgr2=0x7fd28c19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd290b30700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c103f10 0x7fd28c19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd290b30700 1 -- 172.21.15.92:0/2430309893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c1a01b0 con 0x7fd28c102370 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd291331700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c103f10 0x7fd28c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd290b30700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c102370 0x7fd28c198550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd28000b530 tx=0x7fd28000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd289ffb700 1 -- 172.21.15.92:0/2430309893 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280015020 con 0x7fd28c102370 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd289ffb700 1 -- 172.21.15.92:0/2430309893 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd280011d10 con 0x7fd28c102370 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.775+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c1a04a0 con 0x7fd28c102370 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.776+0000 7fd289ffb700 1 -- 172.21.15.92:0/2430309893 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28000f2e0 con 0x7fd28c102370 2024-03-19T01:44:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.776+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1a0ab0 con 0x7fd28c102370 2024-03-19T01:44:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.776+0000 7fd289ffb700 1 -- 172.21.15.92:0/2430309893 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fd280018020 con 0x7fd28c102370 2024-03-19T01:44:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.777+0000 7fd289ffb700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd27c039200 0x7fd27c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:44:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.777+0000 7fd289ffb700 1 -- 172.21.15.92:0/2430309893 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd28004b870 con 0x7fd28c102370 2024-03-19T01:44:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.777+0000 7fd28bfff700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd27c039200 0x7fd27c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:44:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.777+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c068fd0 con 0x7fd28c102370 2024-03-19T01:44:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.777+0000 7fd28bfff700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd27c039200 0x7fd27c03b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd278004630 tx=0x7fd27800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:44:55.782 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.782+0000 7fd289ffb700 1 -- 172.21.15.92:0/2430309893 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd28001f020 con 0x7fd28c102370 2024-03-19T01:44:55.936 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:55 smithi144 podman[68567]: 2024-03-19 01:44:55.579968633 +0000 UTC m=+0.017536351 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:44:55.937 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:55 smithi144 podman[68567]: 2024-03-19 01:44:55.710335581 +0000 UTC m=+0.147903290 container create fbebc8f9da7948db351f533eff3694551f768f42be998d8b7e73ef634c09a440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-19T01:44:55.937 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:55 smithi144 podman[68567]: 2024-03-19 01:44:55.937485173 +0000 UTC m=+0.375052871 container init fbebc8f9da7948db351f533eff3694551f768f42be998d8b7e73ef634c09a440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-19T01:44:55.956 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:55.668+0000 7fd648e1a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:44:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:44:55.990+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd28c062750 con 0x7fd27c039200 2024-03-19T01:44:56.346 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:55 smithi144 podman[68567]: 2024-03-19 01:44:55.940133931 +0000 UTC m=+0.377701628 container start fbebc8f9da7948db351f533eff3694551f768f42be998d8b7e73ef634c09a440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:44:56.346 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:55 smithi144 ceph-mgr[68586]: -- 172.21.15.144:0/469063114 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d960ec5a0 con 0x558d960c0800 2024-03-19T01:44:56.346 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:56 smithi144 bash[68567]: fbebc8f9da7948db351f533eff3694551f768f42be998d8b7e73ef634c09a440 2024-03-19T01:44:56.346 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:56 smithi144 systemd[1]: Started Ceph mgr.y for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:44:56.705 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:56.653+0000 7fd648e1a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:44:56.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-mon[67110]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:56.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:44:56.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:44:56.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:56.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:56.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:56 smithi092 ceph-mon[62574]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:56.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:44:56.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:44:56.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:56.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.708 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:56.846 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:56.498+0000 7fbfb62eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-19T01:44:56.846 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:56.595+0000 7fbfb62eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-19T01:44:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-mon[67199]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:44:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:44:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:44:57.149 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:56 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:56.801+0000 7fd648e1a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:44:57.441 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:57.161+0000 7fd648e1a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:44:57.441 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:57.250+0000 7fd648e1a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:44:57.441 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:57.440+0000 7fd648e1a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:44:57.535 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:57 smithi092 ceph-mon[62574]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:57.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:57 smithi144 ceph-mon[67199]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:57.596 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:57.284+0000 7fbfb62eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-19T01:44:57.596 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:57.376+0000 7fbfb62eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-19T01:44:57.596 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:57.527+0000 7fbfb62eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-19T01:44:57.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:57 smithi159 ceph-mon[67110]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:44:57.705 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:57.633+0000 7fd648e1a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:44:57.970 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:57.623+0000 7fbfb62eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-19T01:44:57.970 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:57.708+0000 7fbfb62eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-19T01:44:57.971 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:57.970+0000 7fbfb62eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-19T01:44:58.178 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:57.916+0000 7fd648e1a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:44:58.257 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:58.078+0000 7fbfb62eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-19T01:44:58.257 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:58.163+0000 7fbfb62eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-19T01:44:58.257 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:58.256+0000 7fbfb62eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-19T01:44:58.455 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:58.177+0000 7fd648e1a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:44:58.455 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:58.411+0000 7fd648e1a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:44:58.528 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:58.431+0000 7fbfb62eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-19T01:44:58.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:44:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:58.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:58.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:58.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:58.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:44:58.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:44:58.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:44:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:58.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:44:58.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:58.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-19T01:44:58.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-19T01:44:58.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:44:59.455 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:59.202+0000 7fd648e1a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:44:59.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:59 smithi092 ceph-mon[62574]: Reconfiguring mgr.x (unknown last config time)... 2024-03-19T01:44:59.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:59 smithi092 ceph-mon[62574]: Reconfiguring daemon mgr.x on smithi092 2024-03-19T01:44:59.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:59 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/696678306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42923908-6c69-4faf-8e1d-36ad4aa578dc"}]: dispatch 2024-03-19T01:44:59.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:59 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/696678306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42923908-6c69-4faf-8e1d-36ad4aa578dc"}]': finished 2024-03-19T01:44:59.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:59 smithi092 ceph-mon[62574]: osdmap e5: 1 total, 0 up, 1 in 2024-03-19T01:44:59.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:59 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:44:59.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:59 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3909013613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:44:59.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:44:59 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:59.846 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:59.398+0000 7fbfb62eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-19T01:44:59.846 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:59.526+0000 7fbfb62eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-19T01:44:59.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-mon[67199]: Reconfiguring mgr.x (unknown last config time)... 2024-03-19T01:44:59.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-mon[67199]: Reconfiguring daemon mgr.x on smithi092 2024-03-19T01:44:59.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/696678306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42923908-6c69-4faf-8e1d-36ad4aa578dc"}]: dispatch 2024-03-19T01:44:59.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/696678306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42923908-6c69-4faf-8e1d-36ad4aa578dc"}]': finished 2024-03-19T01:44:59.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-mon[67199]: osdmap e5: 1 total, 0 up, 1 in 2024-03-19T01:44:59.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:44:59.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3909013613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:44:59.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:44:59.866 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:59.771+0000 7fd648e1a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:44:59.866 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:44:59.866+0000 7fd648e1a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:44:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-mon[67110]: Reconfiguring mgr.x (unknown last config time)... 2024-03-19T01:44:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-mon[67110]: Reconfiguring daemon mgr.x on smithi092 2024-03-19T01:44:59.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/696678306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42923908-6c69-4faf-8e1d-36ad4aa578dc"}]: dispatch 2024-03-19T01:44:59.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/696678306' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42923908-6c69-4faf-8e1d-36ad4aa578dc"}]': finished 2024-03-19T01:44:59.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-mon[67110]: osdmap e5: 1 total, 0 up, 1 in 2024-03-19T01:44:59.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:44:59.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3909013613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:44:59.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:44:59 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:00.140 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:59.860+0000 7fbfb62eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-19T01:45:00.140 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:44:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:44:59.948+0000 7fbfb62eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-19T01:45:00.140 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:45:00.139+0000 7fbfb62eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:45:00.205 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 01:45:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z[68978]: 2024-03-19T01:45:00.076+0000 7fd648e1a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:45:00.530 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:45:00.320+0000 7fbfb62eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-19T01:45:00.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:00.683+0000 7fd289ffb700 1 -- 172.21.15.92:0/2430309893 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 14) ==== 91698+0+0 (secure 0 0 0) 0x7fd28001ed10 con 0x7fd28c102370 2024-03-19T01:45:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:00 smithi092 ceph-mon[62574]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:00 smithi092 ceph-mon[62574]: Standby manager daemon z started 2024-03-19T01:45:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:00 smithi092 ceph-mon[62574]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T01:45:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:00 smithi092 ceph-mon[62574]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:45:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:00 smithi092 ceph-mon[62574]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T01:45:00.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:00 smithi092 ceph-mon[62574]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:45:00.846 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:45:00.602+0000 7fbfb62eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-19T01:45:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-mon[67199]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:00.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-mon[67199]: Standby manager daemon z started 2024-03-19T01:45:00.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-mon[67199]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T01:45:00.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-mon[67199]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:45:00.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-mon[67199]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T01:45:00.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-mon[67199]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:45:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:00 smithi159 ceph-mon[67110]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:00 smithi159 ceph-mon[67110]: Standby manager daemon z started 2024-03-19T01:45:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:00 smithi159 ceph-mon[67110]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-19T01:45:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:00 smithi159 ceph-mon[67110]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:45:00.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:00 smithi159 ceph-mon[67110]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-19T01:45:00.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:00 smithi159 ceph-mon[67110]: from='mgr.? 172.21.15.159:0/726192688' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:45:01.346 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:45:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:45:00.865+0000 7fbfb62eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-19T01:45:01.346 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:45:01 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:45:01.100+0000 7fbfb62eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-19T01:45:01.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:01 smithi144 ceph-mon[67199]: mgrmap e14: x(active, since 81s), standbys: z 2024-03-19T01:45:01.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T01:45:01.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:01 smithi144 ceph-mon[67199]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:01.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:01 smithi159 ceph-mon[67110]: mgrmap e14: x(active, since 81s), standbys: z 2024-03-19T01:45:01.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T01:45:01.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:01 smithi159 ceph-mon[67110]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:02.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:01 smithi092 ceph-mon[62574]: mgrmap e14: x(active, since 81s), standbys: z 2024-03-19T01:45:02.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-19T01:45:02.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:01 smithi092 ceph-mon[62574]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:02.346 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:45:01 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:45:01.896+0000 7fbfb62eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-19T01:45:02.771 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:45:02 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:45:02.459+0000 7fbfb62eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-19T01:45:02.771 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:45:02 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:45:02.555+0000 7fbfb62eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-19T01:45:02.771 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 01:45:02 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y[68582]: 2024-03-19T01:45:02.770+0000 7fbfb62eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-19T01:45:02.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:02.842+0000 7fd289ffb700 1 -- 172.21.15.92:0/2430309893 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7fd280021c20 con 0x7fd28c102370 2024-03-19T01:45:02.919 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:02 smithi092 ceph-mon[62574]: Standby manager daemon y started 2024-03-19T01:45:02.919 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:02 smithi092 ceph-mon[62574]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T01:45:02.919 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:02 smithi092 ceph-mon[62574]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:45:02.919 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:02 smithi092 ceph-mon[62574]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T01:45:02.919 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:02 smithi092 ceph-mon[62574]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:45:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:02 smithi144 ceph-mon[67199]: Standby manager daemon y started 2024-03-19T01:45:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:02 smithi144 ceph-mon[67199]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T01:45:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:02 smithi144 ceph-mon[67199]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:45:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:02 smithi144 ceph-mon[67199]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T01:45:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:02 smithi144 ceph-mon[67199]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:45:03.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:02 smithi159 ceph-mon[67110]: Standby manager daemon y started 2024-03-19T01:45:03.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:02 smithi159 ceph-mon[67110]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-19T01:45:03.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:02 smithi159 ceph-mon[67110]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-19T01:45:03.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:02 smithi159 ceph-mon[67110]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-19T01:45:03.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:02 smithi159 ceph-mon[67110]: from='mgr.? 172.21.15.144:0/3973153283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-19T01:45:04.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:03 smithi144 ceph-mon[67199]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-19T01:45:04.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:03 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T01:45:04.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:03 smithi144 ceph-mon[67199]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:04.124 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:03 smithi092 ceph-mon[62574]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-19T01:45:04.124 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T01:45:04.124 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:03 smithi092 ceph-mon[62574]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:04.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:03 smithi159 ceph-mon[67110]: mgrmap e15: x(active, since 83s), standbys: z, y 2024-03-19T01:45:04.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:03 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-19T01:45:04.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:03 smithi159 ceph-mon[67110]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:05.238 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:05 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:05.239 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:05 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:05.239 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:05 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:05.239 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:05 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:05.239 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:05 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:05.239 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:05 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:05.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:05 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:05.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:05 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:05.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:05 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:05.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:05 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:05.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:05 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:05.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:05 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:05.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:05 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:05.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:05 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:05.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:05 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:05.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:05 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:05.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:05 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:05.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:05 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:06.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:06 smithi092 ceph-mon[62574]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:06.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:06 smithi159 ceph-mon[67110]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:06.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:06 smithi144 ceph-mon[67199]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:07.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-19T01:45:07.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:07.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-19T01:45:07.472 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:07.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-19T01:45:07.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:08.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:08 smithi159 ceph-mon[67110]: Deploying daemon osd.0 on smithi092 2024-03-19T01:45:08.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:08 smithi159 ceph-mon[67110]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:08.535 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:08 smithi092 ceph-mon[62574]: Deploying daemon osd.0 on smithi092 2024-03-19T01:45:08.535 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:08 smithi092 ceph-mon[62574]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:08.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:08 smithi144 ceph-mon[67199]: Deploying daemon osd.0 on smithi092 2024-03-19T01:45:08.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:08 smithi144 ceph-mon[67199]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:10.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:10 smithi092 ceph-mon[62574]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:10.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:10 smithi159 ceph-mon[67110]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:10.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:10 smithi144 ceph-mon[67199]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:12.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:12 smithi159 ceph-mon[67110]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:12.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:12.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:12.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:12.792 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:12 smithi092 ceph-mon[62574]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:12.792 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:12.792 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:12.792 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:12 smithi144 ceph-mon[67199]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:14.633 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:14 smithi092 ceph-mon[62574]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:14.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:14 smithi159 ceph-mon[67110]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:14 smithi144 ceph-mon[67199]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:15.395 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 0 on host 'smithi092' 2024-03-19T01:45:15.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.394+0000 7fd289ffb700 1 -- 172.21.15.92:0/2430309893 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd28c062750 con 0x7fd27c039200 2024-03-19T01:45:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd27c039200 msgr2=0x7fd27c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd27c039200 0x7fd27c03b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd278004630 tx=0x7fd27800b3c0 comp rx=0 tx=0).stop 2024-03-19T01:45:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c102370 msgr2=0x7fd28c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c102370 0x7fd28c198550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd28000b530 tx=0x7fd28000b840 comp rx=0 tx=0).stop 2024-03-19T01:45:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 shutdown_connections 2024-03-19T01:45:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd27c039200 0x7fd27c03b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:15.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd28c103f10 0x7fd28c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:15.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd28c102370 0x7fd28c198550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:15.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 --2- 172.21.15.92:0/2430309893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd28c103570 0x7fd28c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:15.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 >> 172.21.15.92:0/2430309893 conn(0x7fd28c0fd910 msgr2=0x7fd28c1067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:45:15.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 shutdown_connections 2024-03-19T01:45:15.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:45:15.397+0000 7fd292d94700 1 -- 172.21.15.92:0/2430309893 wait complete. 2024-03-19T01:45:16.122 DEBUG:teuthology.orchestra.run.smithi092:osd.0> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.0.service 2024-03-19T01:45:16.124 INFO:tasks.cephadm:Deploying osd.1 on smithi144 with /dev/vg_nvme/lv_4... 2024-03-19T01:45:16.124 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-19T01:45:16.369 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:45:16.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:16 smithi144 ceph-mon[67199]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:16.491 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:16 smithi144 ceph-mon[67199]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-19T01:45:16.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.492 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:16 smithi159 ceph-mon[67110]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:16.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:16 smithi159 ceph-mon[67110]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-19T01:45:16.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.808 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.808 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.808 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:16 smithi092 ceph-mon[62574]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:16.808 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:16 smithi092 ceph-mon[62574]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-19T01:45:16.808 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:16.808 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:17.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:17 smithi092 ceph-mon[62574]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-19T01:45:17.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:17 smithi092 ceph-mon[62574]: osdmap e6: 1 total, 0 up, 1 in 2024-03-19T01:45:17.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:17 smithi092 ceph-mon[62574]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:45:17.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:17 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:17.697 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:17 smithi144 ceph-mon[67199]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-19T01:45:17.697 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:17 smithi144 ceph-mon[67199]: osdmap e6: 1 total, 0 up, 1 in 2024-03-19T01:45:17.697 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:17 smithi144 ceph-mon[67199]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:45:17.697 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:17 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:17.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:17 smithi159 ceph-mon[67110]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-19T01:45:17.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:17 smithi159 ceph-mon[67110]: osdmap e6: 1 total, 0 up, 1 in 2024-03-19T01:45:17.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:17 smithi159 ceph-mon[67110]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:45:17.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:17 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:18.114 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:45:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0[77338]: 2024-03-19T01:45:17.818+0000 7f84d7420700 -1 osd.0 0 waiting for initial osdmap 2024-03-19T01:45:18.114 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:45:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0[77338]: 2024-03-19T01:45:17.822+0000 7f84cf20b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:45:18.269 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-19T01:45:18.269 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-19T01:45:18.269 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-19T01:45:18.269 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-19T01:45:18.269 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0139792 s, 750 MB/s 2024-03-19T01:45:18.269 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-19T01:45:18.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:18 smithi159 ceph-mon[67110]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:18.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:18 smithi159 ceph-mon[67110]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:45:18.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:18 smithi159 ceph-mon[67110]: osdmap e7: 1 total, 0 up, 1 in 2024-03-19T01:45:18.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:18.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:18.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:18 smithi159 ceph-mon[67110]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' 2024-03-19T01:45:18.809 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:18 smithi144 ceph-mon[67199]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:18.810 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:18 smithi144 ceph-mon[67199]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:45:18.810 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:18 smithi144 ceph-mon[67199]: osdmap e7: 1 total, 0 up, 1 in 2024-03-19T01:45:18.810 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:18.810 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:18.810 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:18 smithi144 ceph-mon[67199]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' 2024-03-19T01:45:18.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:18 smithi092 ceph-mon[62574]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:18.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:18 smithi092 ceph-mon[62574]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:45:18.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:18 smithi092 ceph-mon[62574]: osdmap e7: 1 total, 0 up, 1 in 2024-03-19T01:45:18.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:18.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:18.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:18 smithi092 ceph-mon[62574]: from='osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319]' entity='osd.0' 2024-03-19T01:45:18.838 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-03-19T01:45:19.080 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:45:19.487 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:19 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:45:19.487 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:19 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:45:19.487 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:19.487 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:19 smithi144 ceph-mon[67199]: osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] boot 2024-03-19T01:45:19.487 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:19 smithi144 ceph-mon[67199]: osdmap e8: 1 total, 1 up, 1 in 2024-03-19T01:45:19.488 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:19.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:19 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:45:19.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:19 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:45:19.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:19.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:19 smithi092 ceph-mon[62574]: osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] boot 2024-03-19T01:45:19.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:19 smithi092 ceph-mon[62574]: osdmap e8: 1 total, 1 up, 1 in 2024-03-19T01:45:19.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:19.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:19 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:45:19.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:19 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:45:19.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:19.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:19 smithi159 ceph-mon[67110]: osd.0 [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] boot 2024-03-19T01:45:19.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:19 smithi159 ceph-mon[67110]: osdmap e8: 1 total, 1 up, 1 in 2024-03-19T01:45:19.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-19T01:45:20.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.629+0000 7f115ce65700 1 -- 172.21.15.144:0/1399364864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 msgr2=0x7f1158101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:20.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.629+0000 7f115ce65700 1 --2- 172.21.15.144:0/1399364864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 0x7f1158101660 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11480099c0 tx=0x7f1148009cd0 comp rx=0 tx=0).stop 2024-03-19T01:45:20.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.629+0000 7f115ce65700 1 -- 172.21.15.144:0/1399364864 shutdown_connections 2024-03-19T01:45:20.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.629+0000 7f115ce65700 1 --2- 172.21.15.144:0/1399364864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158102de0 0x7f1158109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:20.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.629+0000 7f115ce65700 1 --2- 172.21.15.144:0/1399364864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 0x7f1158101660 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.629+0000 7f115ce65700 1 --2- 172.21.15.144:0/1399364864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1158102440 0x7f11581028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.629+0000 7f115ce65700 1 -- 172.21.15.144:0/1399364864 >> 172.21.15.144:0/1399364864 conn(0x7f11580fc800 msgr2=0x7f11580fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.630+0000 7f115ce65700 1 -- 172.21.15.144:0/1399364864 shutdown_connections 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.630+0000 7f115ce65700 1 -- 172.21.15.144:0/1399364864 wait complete. 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.630+0000 7f115ce65700 1 Processor -- start 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.630+0000 7f115ce65700 1 -- start start 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.630+0000 7f115ce65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 0x7f1158197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115ce65700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1158102440 0x7f11581979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115ce65700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158102de0 0x7f115819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115ce65700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f115810ce30 con 0x7f1158102440 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115ce65700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f115810ccb0 con 0x7f1158101240 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115ce65700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f115810cfb0 con 0x7f1158102de0 2024-03-19T01:45:20.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 0x7f1158197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 0x7f1158197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33528/0 (socket says 172.21.15.144:33528) 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115659c700 1 -- 172.21.15.144:0/328784576 learned_addr learned my addr 172.21.15.144:0/328784576 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f1155d9b700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1158102440 0x7f11581979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f1156d9d700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158102de0 0x7f115819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115659c700 1 -- 172.21.15.144:0/328784576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158102de0 msgr2=0x7f115819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115659c700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158102de0 0x7f115819ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115659c700 1 -- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1158102440 msgr2=0x7f11581979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115659c700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1158102440 0x7f11581979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115659c700 1 -- 172.21.15.144:0/328784576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115819f0d0 con 0x7f1158101240 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115659c700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 0x7f1158197470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f114800fa80 tx=0x7f114800fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f11477fe700 1 -- 172.21.15.144:0/328784576 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148027070 con 0x7f1158101240 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f11477fe700 1 -- 172.21.15.144:0/328784576 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1148022950 con 0x7f1158101240 2024-03-19T01:45:20.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115819f360 con 0x7f1158101240 2024-03-19T01:45:20.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f11477fe700 1 -- 172.21.15.144:0/328784576 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148009e60 con 0x7f1158101240 2024-03-19T01:45:20.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.631+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115819f840 con 0x7f1158101240 2024-03-19T01:45:20.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.632+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1158068fd0 con 0x7f1158101240 2024-03-19T01:45:20.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.633+0000 7f11477fe700 1 -- 172.21.15.144:0/328784576 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f114802a070 con 0x7f1158101240 2024-03-19T01:45:20.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.634+0000 7f11477fe700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f11400a2800 0x7f11400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:45:20.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.634+0000 7f11477fe700 1 -- 172.21.15.144:0/328784576 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f11480cdc50 con 0x7f1158101240 2024-03-19T01:45:20.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.634+0000 7f1155d9b700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f11400a2800 0x7f11400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:45:20.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.635+0000 7f1155d9b700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f11400a2800 0x7f11400a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f114c005950 tx=0x7f114c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:45:20.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.639+0000 7f11477fe700 1 -- 172.21.15.144:0/328784576 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1148013070 con 0x7f1158101240 2024-03-19T01:45:20.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:20.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:45:20.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:20.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:20.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: osdmap e9: 1 total, 1 up, 1 in 2024-03-19T01:45:20.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:45:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:20.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: osdmap e9: 1 total, 1 up, 1 in 2024-03-19T01:45:20.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:20.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:20.842+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1158062750 con 0x7f11400a2800 2024-03-19T01:45:20.842 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-19T01:45:20.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:45:20.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:20.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:20.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:20.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: osdmap e9: 1 total, 1 up, 1 in 2024-03-19T01:45:20.843 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:21.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:21 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi092 2024-03-19T01:45:21.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:21 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi092 to 17088M 2024-03-19T01:45:21.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:21 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:21.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:21 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:21.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:21 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:21.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:21 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:45:21.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:21 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:45:21.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:21 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:21.729 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:21 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi092 2024-03-19T01:45:21.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:21 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi092 to 17088M 2024-03-19T01:45:21.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:21 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:21.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:21 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:21.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:21 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:21.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:21 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:45:21.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:21 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:45:21.730 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:21 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:21.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:21 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi092 2024-03-19T01:45:21.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:21 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi092 to 17088M 2024-03-19T01:45:21.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:21 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:21.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:21 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:21.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:21 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:21.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:21 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:45:21.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:21 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:45:21.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:21 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:22.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:22 smithi144 ceph-mon[67199]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:45:22.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:22 smithi144 ceph-mon[67199]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:22.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:22 smithi159 ceph-mon[67110]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:45:22.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:22 smithi159 ceph-mon[67110]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:22.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:22 smithi092 ceph-mon[62574]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:45:22.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:22 smithi092 ceph-mon[62574]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:24.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:24 smithi159 ceph-mon[67110]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:24.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:24 smithi159 ceph-mon[67110]: from='client.? 172.21.15.144:0/2012308587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2481d494-0d38-4da0-a314-002c20836f0c"}]: dispatch 2024-03-19T01:45:24.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:24 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2481d494-0d38-4da0-a314-002c20836f0c"}]: dispatch 2024-03-19T01:45:24.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:24 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2481d494-0d38-4da0-a314-002c20836f0c"}]': finished 2024-03-19T01:45:24.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:24 smithi159 ceph-mon[67110]: osdmap e10: 2 total, 1 up, 2 in 2024-03-19T01:45:24.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:24.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:24 smithi159 ceph-mon[67110]: from='client.? 172.21.15.144:0/1728210327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:45:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:24 smithi092 ceph-mon[62574]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:24 smithi092 ceph-mon[62574]: from='client.? 172.21.15.144:0/2012308587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2481d494-0d38-4da0-a314-002c20836f0c"}]: dispatch 2024-03-19T01:45:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:24 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2481d494-0d38-4da0-a314-002c20836f0c"}]: dispatch 2024-03-19T01:45:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:24 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2481d494-0d38-4da0-a314-002c20836f0c"}]': finished 2024-03-19T01:45:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:24 smithi092 ceph-mon[62574]: osdmap e10: 2 total, 1 up, 2 in 2024-03-19T01:45:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:24.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:24 smithi092 ceph-mon[62574]: from='client.? 172.21.15.144:0/1728210327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:45:24.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:24 smithi144 ceph-mon[67199]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:24.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:24 smithi144 ceph-mon[67199]: from='client.? 172.21.15.144:0/2012308587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2481d494-0d38-4da0-a314-002c20836f0c"}]: dispatch 2024-03-19T01:45:24.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:24 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2481d494-0d38-4da0-a314-002c20836f0c"}]: dispatch 2024-03-19T01:45:24.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:24 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2481d494-0d38-4da0-a314-002c20836f0c"}]': finished 2024-03-19T01:45:24.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:24 smithi144 ceph-mon[67199]: osdmap e10: 2 total, 1 up, 2 in 2024-03-19T01:45:24.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:24.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:24 smithi144 ceph-mon[67199]: from='client.? 172.21.15.144:0/1728210327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:45:26.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:26 smithi159 ceph-mon[67110]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:26.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:26 smithi092 ceph-mon[62574]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:26.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:26 smithi144 ceph-mon[67199]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:28.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:28 smithi144 ceph-mon[67199]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:28.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:28 smithi159 ceph-mon[67110]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:28 smithi092 ceph-mon[62574]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:30.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:30 smithi159 ceph-mon[67110]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:30.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-19T01:45:30.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:30 smithi092 ceph-mon[62574]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-19T01:45:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:30.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:30 smithi144 ceph-mon[67199]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:30.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-19T01:45:30.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:31.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:31 smithi144 ceph-mon[67199]: Deploying daemon osd.1 on smithi144 2024-03-19T01:45:31.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:31 smithi159 ceph-mon[67110]: Deploying daemon osd.1 on smithi144 2024-03-19T01:45:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:31 smithi092 ceph-mon[62574]: Deploying daemon osd.1 on smithi144 2024-03-19T01:45:32.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:32 smithi144 ceph-mon[67199]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:32.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:32 smithi159 ceph-mon[67110]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:32 smithi092 ceph-mon[62574]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:34.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:34 smithi159 ceph-mon[67110]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:34.747 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:34 smithi144 ceph-mon[67199]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:34 smithi092 ceph-mon[62574]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:36.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:36 smithi144 ceph-mon[67199]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:36.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:36 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:36.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:36 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:36.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:36 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:36 smithi092 ceph-mon[62574]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:36 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:36 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:36 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:36 smithi159 ceph-mon[67110]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:36 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:36 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:36 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:38.628 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:38 smithi144 ceph-mon[67199]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:38.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:38 smithi092 ceph-mon[62574]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:38 smithi159 ceph-mon[67110]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:39.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.256+0000 7f11477fe700 1 -- 172.21.15.144:0/328784576 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1158062750 con 0x7f11400a2800 2024-03-19T01:45:39.256 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 1 on host 'smithi144' 2024-03-19T01:45:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f11400a2800 msgr2=0x7f11400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f11400a2800 0x7f11400a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f114c005950 tx=0x7f114c00b410 comp rx=0 tx=0).stop 2024-03-19T01:45:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 msgr2=0x7f1158197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 0x7f1158197470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f114800fa80 tx=0x7f114800fab0 comp rx=0 tx=0).stop 2024-03-19T01:45:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 shutdown_connections 2024-03-19T01:45:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f11400a2800 0x7f11400a4cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158102de0 0x7f115819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:39.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1158101240 0x7f1158197470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:39.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 --2- 172.21.15.144:0/328784576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1158102440 0x7f11581979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:39.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 >> 172.21.15.144:0/328784576 conn(0x7f11580fc800 msgr2=0x7f11581057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:45:39.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 shutdown_connections 2024-03-19T01:45:39.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:45:39.258+0000 7f115ce65700 1 -- 172.21.15.144:0/328784576 wait complete. 2024-03-19T01:45:39.764 DEBUG:teuthology.orchestra.run.smithi144:osd.1> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.1.service 2024-03-19T01:45:39.766 INFO:tasks.cephadm:Deploying osd.2 on smithi159 with /dev/vg_nvme/lv_4... 2024-03-19T01:45:39.766 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-19T01:45:40.011 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:45:40.096 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:45:39 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1[71782]: 2024-03-19T01:45:39.814+0000 7eff1082f7c0 -1 osd.1 0 log_to_monitors true 2024-03-19T01:45:40.412 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.412 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.413 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:40 smithi159 ceph-mon[67110]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:40.413 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.413 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.413 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:40 smithi159 ceph-mon[67110]: from='osd.1 [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:45:40.413 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:40 smithi159 ceph-mon[67110]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:45:40.439 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.440 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.440 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:40 smithi144 ceph-mon[67199]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:40.440 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.440 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.440 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:40 smithi144 ceph-mon[67199]: from='osd.1 [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:45:40.440 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:40 smithi144 ceph-mon[67199]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:45:40.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:40 smithi092 ceph-mon[62574]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:40.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:40.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:40 smithi092 ceph-mon[62574]: from='osd.1 [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:45:40.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:40 smithi092 ceph-mon[62574]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-19T01:45:41.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:41 smithi159 ceph-mon[67110]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-19T01:45:41.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:41 smithi159 ceph-mon[67110]: osdmap e11: 2 total, 1 up, 2 in 2024-03-19T01:45:41.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:41.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:41 smithi159 ceph-mon[67110]: from='osd.1 [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:45:41.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:41 smithi159 ceph-mon[67110]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:45:41.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:41 smithi092 ceph-mon[62574]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-19T01:45:41.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:41 smithi092 ceph-mon[62574]: osdmap e11: 2 total, 1 up, 2 in 2024-03-19T01:45:41.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:41.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:41 smithi092 ceph-mon[62574]: from='osd.1 [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:45:41.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:41 smithi092 ceph-mon[62574]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:45:41.846 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:45:41 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1[71782]: 2024-03-19T01:45:41.504+0000 7eff05e94700 -1 osd.1 0 waiting for initial osdmap 2024-03-19T01:45:41.846 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:45:41 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1[71782]: 2024-03-19T01:45:41.509+0000 7eff01c87700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:45:41.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:41 smithi144 ceph-mon[67199]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-19T01:45:41.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:41 smithi144 ceph-mon[67199]: osdmap e11: 2 total, 1 up, 2 in 2024-03-19T01:45:41.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:41.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:41 smithi144 ceph-mon[67199]: from='osd.1 [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:45:41.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:41 smithi144 ceph-mon[67199]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:45:41.867 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-19T01:45:41.867 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-19T01:45:41.868 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-19T01:45:41.868 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-19T01:45:41.868 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112984 s, 928 MB/s 2024-03-19T01:45:41.868 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-19T01:45:42.441 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2024-03-19T01:45:42.682 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:45:42.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:42 smithi159 ceph-mon[67110]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:42.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:42 smithi159 ceph-mon[67110]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:45:42.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:42 smithi159 ceph-mon[67110]: osdmap e12: 2 total, 1 up, 2 in 2024-03-19T01:45:42.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:42.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:42 smithi092 ceph-mon[62574]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:42 smithi092 ceph-mon[62574]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:45:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:42 smithi092 ceph-mon[62574]: osdmap e12: 2 total, 1 up, 2 in 2024-03-19T01:45:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:42.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:42 smithi144 ceph-mon[67199]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:42.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:42 smithi144 ceph-mon[67199]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:45:42.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:42 smithi144 ceph-mon[67199]: osdmap e12: 2 total, 1 up, 2 in 2024-03-19T01:45:42.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:42.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:43.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: osd.1 [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] boot 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: osdmap e13: 2 total, 2 up, 2 in 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:43.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:43.812 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: osd.1 [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] boot 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: osdmap e13: 2 total, 2 up, 2 in 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:43.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:43.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:45:43.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:45:43.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: osd.1 [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] boot 2024-03-19T01:45:43.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: osdmap e13: 2 total, 2 up, 2 in 2024-03-19T01:45:43.900 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:45:43.901 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:45:44.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/3359266244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98102340 msgr2=0x7f8a98102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:44.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/3359266244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98102340 0x7f8a98102760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009990 tx=0x7f8a88009ca0 comp rx=0 tx=0).stop 2024-03-19T01:45:44.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/3359266244 shutdown_connections 2024-03-19T01:45:44.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/3359266244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a98103ee0 0x7f8a9810a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:44.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/3359266244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98102340 0x7f8a98102760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:44.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/3359266244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a98103540 0x7f8a981039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:44.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/3359266244 >> 172.21.15.159:0/3359266244 conn(0x7f8a980fd920 msgr2=0x7f8a980ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:45:44.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/3359266244 shutdown_connections 2024-03-19T01:45:44.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/3359266244 wait complete. 2024-03-19T01:45:44.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.293+0000 7f8a9d7e6700 1 Processor -- start 2024-03-19T01:45:44.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a9d7e6700 1 -- start start 2024-03-19T01:45:44.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a9d7e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a98102340 0x7f8a980760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:45:44.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a9d7e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a98103540 0x7f8a98074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:45:44.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a9d7e6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98103ee0 0x7f8a98074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:45:44.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a9d7e6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9810df80 con 0x7f8a98102340 2024-03-19T01:45:44.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a9d7e6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9810de00 con 0x7f8a98103540 2024-03-19T01:45:44.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a9d7e6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9810e100 con 0x7f8a98103ee0 2024-03-19T01:45:44.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a977fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98103ee0 0x7f8a98074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:45:44.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a977fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98103ee0 0x7f8a98074c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38458/0 (socket says 172.21.15.159:38458) 2024-03-19T01:45:44.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a977fe700 1 -- 172.21.15.159:0/745782102 learned_addr learned my addr 172.21.15.159:0/745782102 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:45:44.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a8e7fc700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a98103540 0x7f8a98074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:45:44.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a96ffd700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a98102340 0x7f8a980760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:45:44.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a977fe700 1 -- 172.21.15.159:0/745782102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a98103540 msgr2=0x7f8a98074700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:44.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a977fe700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a98103540 0x7f8a98074700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:44.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a977fe700 1 -- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a98102340 msgr2=0x7f8a980760b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:45:44.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a977fe700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a98102340 0x7f8a980760b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:45:44.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a977fe700 1 -- 172.21.15.159:0/745782102 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a980754f0 con 0x7f8a98103ee0 2024-03-19T01:45:44.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.294+0000 7f8a977fe700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98103ee0 0x7f8a98074c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a90007ee0 tx=0x7f8a9000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:45:44.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.295+0000 7f8a94ff9700 1 -- 172.21.15.159:0/745782102 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000dbd0 con 0x7f8a98103ee0 2024-03-19T01:45:44.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.295+0000 7f8a94ff9700 1 -- 172.21.15.159:0/745782102 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9000dd70 con 0x7f8a98103ee0 2024-03-19T01:45:44.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.295+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a981a2780 con 0x7f8a98103ee0 2024-03-19T01:45:44.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.295+0000 7f8a94ff9700 1 -- 172.21.15.159:0/745782102 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90005a60 con 0x7f8a98103ee0 2024-03-19T01:45:44.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.295+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a981a2cc0 con 0x7f8a98103ee0 2024-03-19T01:45:44.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.296+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a98068fd0 con 0x7f8a98103ee0 2024-03-19T01:45:44.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.297+0000 7f8a94ff9700 1 -- 172.21.15.159:0/745782102 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f8a9001a020 con 0x7f8a98103ee0 2024-03-19T01:45:44.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.298+0000 7f8a94ff9700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8a840a2800 0x7f8a840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:45:44.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.298+0000 7f8a94ff9700 1 -- 172.21.15.159:0/745782102 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f8a900cd000 con 0x7f8a98103ee0 2024-03-19T01:45:44.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.298+0000 7f8a96ffd700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8a840a2800 0x7f8a840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:45:44.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.298+0000 7f8a96ffd700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8a840a2800 0x7f8a840a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a880039c0 tx=0x7f8a88005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:45:44.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.302+0000 7f8a94ff9700 1 -- 172.21.15.159:0/745782102 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8a90010070 con 0x7f8a98103ee0 2024-03-19T01:45:44.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:45:44.507+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8a9802dfc0 con 0x7f8a840a2800 2024-03-19T01:45:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:44 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi144 2024-03-19T01:45:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:44 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi144 to 17080M 2024-03-19T01:45:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:44 smithi092 ceph-mon[62574]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:44 smithi092 ceph-mon[62574]: osdmap e14: 2 total, 2 up, 2 in 2024-03-19T01:45:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:45:44.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:45:44.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:44 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi144 2024-03-19T01:45:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:44 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi144 to 17080M 2024-03-19T01:45:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:44 smithi144 ceph-mon[67199]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:44.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:44 smithi144 ceph-mon[67199]: osdmap e14: 2 total, 2 up, 2 in 2024-03-19T01:45:44.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:45:44.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:45:44.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:44 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi144 2024-03-19T01:45:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:44 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi144 to 17080M 2024-03-19T01:45:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:44 smithi159 ceph-mon[67110]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-19T01:45:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:44 smithi159 ceph-mon[67110]: osdmap e14: 2 total, 2 up, 2 in 2024-03-19T01:45:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:45:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:45:44.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:45.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:45 smithi159 ceph-mon[67110]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:45:45.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:45 smithi092 ceph-mon[62574]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:45:45.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:45 smithi144 ceph-mon[67199]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:45:46.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:46 smithi159 ceph-mon[67110]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:46 smithi092 ceph-mon[62574]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:46.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:46 smithi144 ceph-mon[67199]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:47 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/2091998011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6cfae2-9e8c-4588-aa01-0fc24c7297d3"}]: dispatch 2024-03-19T01:45:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:47 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6cfae2-9e8c-4588-aa01-0fc24c7297d3"}]: dispatch 2024-03-19T01:45:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:47 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e6cfae2-9e8c-4588-aa01-0fc24c7297d3"}]': finished 2024-03-19T01:45:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:47 smithi092 ceph-mon[62574]: osdmap e15: 3 total, 2 up, 3 in 2024-03-19T01:45:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:45:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:47 smithi144 ceph-mon[67199]: from='client.? 172.21.15.159:0/2091998011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6cfae2-9e8c-4588-aa01-0fc24c7297d3"}]: dispatch 2024-03-19T01:45:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:47 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6cfae2-9e8c-4588-aa01-0fc24c7297d3"}]: dispatch 2024-03-19T01:45:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:47 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e6cfae2-9e8c-4588-aa01-0fc24c7297d3"}]': finished 2024-03-19T01:45:47.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:47 smithi144 ceph-mon[67199]: osdmap e15: 3 total, 2 up, 3 in 2024-03-19T01:45:47.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:45:47.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:47 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/2091998011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6cfae2-9e8c-4588-aa01-0fc24c7297d3"}]: dispatch 2024-03-19T01:45:47.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:47 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e6cfae2-9e8c-4588-aa01-0fc24c7297d3"}]: dispatch 2024-03-19T01:45:47.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:47 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e6cfae2-9e8c-4588-aa01-0fc24c7297d3"}]': finished 2024-03-19T01:45:47.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:47 smithi159 ceph-mon[67110]: osdmap e15: 3 total, 2 up, 3 in 2024-03-19T01:45:47.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:45:48.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:48 smithi092 ceph-mon[62574]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:48.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:48 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/3163892734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:45:48.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:48 smithi144 ceph-mon[67199]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:48.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:48 smithi144 ceph-mon[67199]: from='client.? 172.21.15.159:0/3163892734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:45:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:48 smithi159 ceph-mon[67110]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:48 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/3163892734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:45:50.800 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:50 smithi159 ceph-mon[67110]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:50 smithi092 ceph-mon[62574]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:50.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:50 smithi144 ceph-mon[67199]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:52.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:52 smithi159 ceph-mon[67110]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:52 smithi092 ceph-mon[62574]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:52.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:52 smithi144 ceph-mon[67199]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:54.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:54 smithi092 ceph-mon[62574]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:54.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-19T01:45:54.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:54 smithi144 ceph-mon[67199]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-19T01:45:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:54.888 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:54 smithi159 ceph-mon[67110]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:54.888 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-19T01:45:54.888 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:45:55.698 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:55 smithi159 ceph-mon[67110]: Deploying daemon osd.2 on smithi159 2024-03-19T01:45:55.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:55 smithi092 ceph-mon[62574]: Deploying daemon osd.2 on smithi159 2024-03-19T01:45:55.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:55 smithi144 ceph-mon[67199]: Deploying daemon osd.2 on smithi159 2024-03-19T01:45:56.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:56 smithi159 ceph-mon[67110]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:56.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:56 smithi092 ceph-mon[62574]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:56.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:56 smithi144 ceph-mon[67199]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:58.787 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:45:58 smithi159 ceph-mon[67110]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:58.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:45:58 smithi092 ceph-mon[62574]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:45:58.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:45:58 smithi144 ceph-mon[67199]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:00.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:00 smithi159 ceph-mon[67110]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:00.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:00.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:00.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:00 smithi092 ceph-mon[62574]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:00.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:00.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:00.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:00 smithi144 ceph-mon[67199]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:02.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:02 smithi092 ceph-mon[62574]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:02.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:02 smithi144 ceph-mon[67199]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:02 smithi159 ceph-mon[67110]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:03.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.076+0000 7f8a94ff9700 1 -- 172.21.15.159:0/745782102 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8a9802dfc0 con 0x7f8a840a2800 2024-03-19T01:46:03.077 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 2 on host 'smithi159' 2024-03-19T01:46:03.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8a840a2800 msgr2=0x7f8a840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:03.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8a840a2800 0x7f8a840a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a880039c0 tx=0x7f8a88005fb0 comp rx=0 tx=0).stop 2024-03-19T01:46:03.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98103ee0 msgr2=0x7f8a98074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98103ee0 0x7f8a98074c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a90007ee0 tx=0x7f8a9000d3b0 comp rx=0 tx=0).stop 2024-03-19T01:46:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 shutdown_connections 2024-03-19T01:46:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8a840a2800 0x7f8a840a4cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a98103ee0 0x7f8a98074c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a98102340 0x7f8a980760b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 --2- 172.21.15.159:0/745782102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a98103540 0x7f8a98074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 >> 172.21.15.159:0/745782102 conn(0x7f8a980fd920 msgr2=0x7f8a98106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:46:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 shutdown_connections 2024-03-19T01:46:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:03.079+0000 7f8a9d7e6700 1 -- 172.21.15.159:0/745782102 wait complete. 2024-03-19T01:46:03.769 DEBUG:teuthology.orchestra.run.smithi159:osd.2> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.2.service 2024-03-19T01:46:03.771 INFO:tasks.cephadm:Deploying osd.3 on smithi092 with /dev/vg_nvme/lv_3... 2024-03-19T01:46:03.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-19T01:46:04.028 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:46:04.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:04 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:04 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:04 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:04 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:04 smithi092 ceph-mon[62574]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:04.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:04 smithi092 ceph-mon[62574]: from='osd.2 [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:46:04.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:04 smithi092 ceph-mon[62574]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:46:04.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:04 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:04 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:04 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:04 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:04 smithi159 ceph-mon[67110]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:04.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:04 smithi159 ceph-mon[67110]: from='osd.2 [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:46:04.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:04 smithi159 ceph-mon[67110]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:46:04.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:04 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:04 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:04 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:04 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:04.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:04 smithi144 ceph-mon[67199]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:04.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:04 smithi144 ceph-mon[67199]: from='osd.2 [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:46:04.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:04 smithi144 ceph-mon[67199]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-19T01:46:05.389 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:05 smithi159 ceph-mon[67110]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-19T01:46:05.390 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:05 smithi159 ceph-mon[67110]: osdmap e16: 3 total, 2 up, 3 in 2024-03-19T01:46:05.390 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:05 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:05.390 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:05 smithi159 ceph-mon[67110]: from='osd.2 [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:46:05.390 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:05 smithi159 ceph-mon[67110]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:46:05.465 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:05 smithi092 ceph-mon[62574]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-19T01:46:05.465 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:05 smithi092 ceph-mon[62574]: osdmap e16: 3 total, 2 up, 3 in 2024-03-19T01:46:05.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:05 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:05.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:05 smithi092 ceph-mon[62574]: from='osd.2 [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:46:05.466 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:05 smithi092 ceph-mon[62574]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:46:05.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:05 smithi144 ceph-mon[67199]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-19T01:46:05.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:05 smithi144 ceph-mon[67199]: osdmap e16: 3 total, 2 up, 3 in 2024-03-19T01:46:05.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:05 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:05.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:05 smithi144 ceph-mon[67199]: from='osd.2 [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:46:05.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:05 smithi144 ceph-mon[67199]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:46:05.705 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2[72251]: 2024-03-19T01:46:05.568+0000 7fa47b9da700 -1 osd.2 0 waiting for initial osdmap 2024-03-19T01:46:05.705 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2[72251]: 2024-03-19T01:46:05.575+0000 7fa4737c5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:46:06.181 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-19T01:46:06.181 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-19T01:46:06.181 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-03-19T01:46:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-03-19T01:46:06.182 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115298 s, 909 MB/s 2024-03-19T01:46:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-03-19T01:46:06.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:06 smithi159 ceph-mon[67110]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:46:06.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:06 smithi159 ceph-mon[67110]: osdmap e17: 3 total, 2 up, 3 in 2024-03-19T01:46:06.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:06 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:06.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:06 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:06.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:06 smithi159 ceph-mon[67110]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:06.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:06 smithi159 ceph-mon[67110]: from='osd.2 ' entity='osd.2' 2024-03-19T01:46:06.481 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:06 smithi092 ceph-mon[62574]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:46:06.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:06 smithi092 ceph-mon[62574]: osdmap e17: 3 total, 2 up, 3 in 2024-03-19T01:46:06.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:06 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:06.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:06 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:06.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:06 smithi092 ceph-mon[62574]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:06.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:06 smithi092 ceph-mon[62574]: from='osd.2 ' entity='osd.2' 2024-03-19T01:46:06.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:06 smithi144 ceph-mon[67199]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:46:06.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:06 smithi144 ceph-mon[67199]: osdmap e17: 3 total, 2 up, 3 in 2024-03-19T01:46:06.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:06 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:06.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:06 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:06.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:06 smithi144 ceph-mon[67199]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-19T01:46:06.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:06 smithi144 ceph-mon[67199]: from='osd.2 ' entity='osd.2' 2024-03-19T01:46:06.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-03-19T01:46:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:46:07.430 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: osd.2 [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] boot 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: osdmap e18: 3 total, 3 up, 3 in 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:07.431 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.432 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:07.432 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:07.432 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:07.432 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:46:07.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:46:07.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:07.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: osd.2 [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] boot 2024-03-19T01:46:07.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: osdmap e18: 3 total, 3 up, 3 in 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:07.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:46:07.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: osd.2 [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] boot 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: osdmap e18: 3 total, 3 up, 3 in 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:07.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:07.598 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:08.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:08 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi159 2024-03-19T01:46:08.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:08 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi159 to 17080M 2024-03-19T01:46:08.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:08 smithi159 ceph-mon[67110]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:08.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:46:08.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-19T01:46:08.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:08 smithi159 ceph-mon[67110]: osdmap e19: 3 total, 3 up, 3 in 2024-03-19T01:46:08.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:46:08.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:08 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi159 2024-03-19T01:46:08.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:08 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi159 to 17080M 2024-03-19T01:46:08.486 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:08 smithi092 ceph-mon[62574]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:08.487 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:46:08.487 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-19T01:46:08.487 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:08 smithi092 ceph-mon[62574]: osdmap e19: 3 total, 3 up, 3 in 2024-03-19T01:46:08.487 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:46:08.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:08 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi159 2024-03-19T01:46:08.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:08 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi159 to 17080M 2024-03-19T01:46:08.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:08 smithi144 ceph-mon[67199]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:08.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:46:08.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-19T01:46:08.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:08 smithi144 ceph-mon[67199]: osdmap e19: 3 total, 3 up, 3 in 2024-03-19T01:46:08.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T01:46:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.707+0000 7fbee5b9b700 1 -- 172.21.15.92:0/799157738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0103570 msgr2=0x7fbee01039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.707+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/799157738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0103570 0x7fbee01039d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbed40099c0 tx=0x7fbed4009cd0 comp rx=0 tx=0).stop 2024-03-19T01:46:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.707+0000 7fbee5b9b700 1 -- 172.21.15.92:0/799157738 shutdown_connections 2024-03-19T01:46:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.707+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/799157738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbee0103f10 0x7fbee010a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.707+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/799157738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbee0102370 0x7fbee0102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.707+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/799157738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0103570 0x7fbee01039d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.707+0000 7fbee5b9b700 1 -- 172.21.15.92:0/799157738 >> 172.21.15.92:0/799157738 conn(0x7fbee00fd8f0 msgr2=0x7fbee00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:46:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.707+0000 7fbee5b9b700 1 -- 172.21.15.92:0/799157738 shutdown_connections 2024-03-19T01:46:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.707+0000 7fbee5b9b700 1 -- 172.21.15.92:0/799157738 wait complete. 2024-03-19T01:46:08.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbee5b9b700 1 Processor -- start 2024-03-19T01:46:08.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbee5b9b700 1 -- start start 2024-03-19T01:46:08.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbee5b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0102370 0x7fbee0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:08.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbee5b9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbee0103570 0x7fbee0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbee5b9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbee0103f10 0x7fbee019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbee5b9b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbee010dfb0 con 0x7fbee0102370 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbee5b9b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbee010de30 con 0x7fbee0103570 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbee5b9b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbee010e130 con 0x7fbee0103f10 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbedf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0102370 0x7fbee0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbedf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0102370 0x7fbee0198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36520/0 (socket says 172.21.15.92:36520) 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbedf7fe700 1 -- 172.21.15.92:0/2755811577 learned_addr learned my addr 172.21.15.92:0/2755811577 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbedffff700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbee0103f10 0x7fbee019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.708+0000 7fbedeffd700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbee0103570 0x7fbee0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbedf7fe700 1 -- 172.21.15.92:0/2755811577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbee0103f10 msgr2=0x7fbee019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbedf7fe700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbee0103f10 0x7fbee019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbedf7fe700 1 -- 172.21.15.92:0/2755811577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbee0103570 msgr2=0x7fbee0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:08.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbedf7fe700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbee0103570 0x7fbee0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbedf7fe700 1 -- 172.21.15.92:0/2755811577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee01a01a0 con 0x7fbee0102370 2024-03-19T01:46:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbedf7fe700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0102370 0x7fbee0198540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbed000e9a0 tx=0x7fbed000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:46:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbedcff9700 1 -- 172.21.15.92:0/2755811577 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0004220 con 0x7fbee0102370 2024-03-19T01:46:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee01a0490 con 0x7fbee0102370 2024-03-19T01:46:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbedcff9700 1 -- 172.21.15.92:0/2755811577 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed00043c0 con 0x7fbee0102370 2024-03-19T01:46:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbedcff9700 1 -- 172.21.15.92:0/2755811577 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0003df0 con 0x7fbee0102370 2024-03-19T01:46:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.709+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee01a0a20 con 0x7fbee0102370 2024-03-19T01:46:08.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.710+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee0068fd0 con 0x7fbee0102370 2024-03-19T01:46:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.711+0000 7fbedcff9700 1 -- 172.21.15.92:0/2755811577 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7fbed000c810 con 0x7fbee0102370 2024-03-19T01:46:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.712+0000 7fbedcff9700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbec80a2800 0x7fbec80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.712+0000 7fbedeffd700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbec80a2800 0x7fbec80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.712+0000 7fbedcff9700 1 -- 172.21.15.92:0/2755811577 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7fbed00cd810 con 0x7fbee0102370 2024-03-19T01:46:08.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.712+0000 7fbedeffd700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbec80a2800 0x7fbec80a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbed4000c00 tx=0x7fbed4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:46:08.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.716+0000 7fbedcff9700 1 -- 172.21.15.92:0/2755811577 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbed009bdf0 con 0x7fbee0102370 2024-03-19T01:46:08.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:08.938+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbee0062750 con 0x7fbec80a2800 2024-03-19T01:46:09.310 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:46:09 smithi092 sudo[81663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:46:09.310 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:46:09 smithi092 sudo[81663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:09.311 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:46:09 smithi092 sudo[81663]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:09.311 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:46:09 smithi092 sudo[81663]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:09.596 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:46:09 smithi092 sudo[81666]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:46:09.597 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:46:09 smithi092 sudo[81666]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:09.597 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:46:09 smithi092 sudo[81666]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:09.597 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 01:46:09 smithi092 sudo[81666]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:09.832 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:46:09 smithi144 sudo[75918]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:46:09.832 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:46:09 smithi144 sudo[75918]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:09.832 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:46:09 smithi144 sudo[75918]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:09.832 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:46:09 smithi144 sudo[75918]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:09.832 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-19T01:46:09.833 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:09 smithi144 ceph-mon[67199]: osdmap e20: 3 total, 3 up, 3 in 2024-03-19T01:46:09.833 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:09 smithi144 ceph-mon[67199]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:46:09.833 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:46:09.833 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:46:09.833 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:09.833 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:09.833 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:09 smithi144 ceph-mon[67199]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:09.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-19T01:46:09.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:09 smithi092 ceph-mon[62574]: osdmap e20: 3 total, 3 up, 3 in 2024-03-19T01:46:09.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:09 smithi092 ceph-mon[62574]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:46:09.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:46:09.899 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:46:09.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:09.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:09.900 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:09 smithi092 ceph-mon[62574]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:09.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-19T01:46:09.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:09 smithi159 ceph-mon[67110]: osdmap e20: 3 total, 3 up, 3 in 2024-03-19T01:46:09.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:09 smithi159 ceph-mon[67110]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:46:09.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:46:09.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:46:09.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:09.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:09.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:09 smithi159 ceph-mon[67110]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:10.096 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:46:09 smithi144 sudo[75921]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:46:10.097 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:46:09 smithi144 sudo[75921]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:10.097 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:46:09 smithi144 sudo[75921]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:10.097 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 01:46:09 smithi144 sudo[75921]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:10.357 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:10 smithi159 sudo[76309]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:46:10.357 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:10 smithi159 sudo[76309]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:10.357 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:10 smithi159 sudo[76309]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:10.357 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:10 smithi159 sudo[76309]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:10.706 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:10 smithi159 sudo[76312]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:46:10.706 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:10 smithi159 sudo[76312]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:10.706 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:10 smithi159 sudo[76312]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:10.706 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 01:46:10 smithi159 sudo[76312]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:10.894 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 sudo[81749]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:46:10.894 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 sudo[81749]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:10.894 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 sudo[81749]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:10.894 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 sudo[81749]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:10.911+0000 7fbedcff9700 1 -- 172.21.15.92:0/2755811577 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fbed000cb00 con 0x7fbee0102370 2024-03-19T01:46:11.152 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:10 smithi159 ceph-mon[67110]: osdmap e21: 3 total, 3 up, 3 in 2024-03-19T01:46:11.152 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:11.152 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:11.152 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:10 smithi159 ceph-mon[67110]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:46:11.152 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 sudo[76315]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:46:11.153 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 sudo[76315]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:11.153 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 sudo[76315]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:11.153 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 sudo[76315]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 sudo[81771]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:46:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 sudo[81771]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 sudo[81771]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 sudo[81771]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 ceph-mon[62574]: osdmap e21: 3 total, 3 up, 3 in 2024-03-19T01:46:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:11.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:10 smithi092 ceph-mon[62574]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:46:11.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:10 smithi144 ceph-mon[67199]: osdmap e21: 3 total, 3 up, 3 in 2024-03-19T01:46:11.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:11.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:11.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:10 smithi144 ceph-mon[67199]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:46:11.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 sudo[76318]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:46:11.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 sudo[76318]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:11.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 sudo[76318]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:11.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 sudo[76318]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:11.880 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 sudo[75924]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-19T01:46:11.880 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 sudo[75924]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:11.881 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 sudo[75924]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:11.881 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 sudo[75924]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:12.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:46:12.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:46:12.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:46:12.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:46:12.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:46:12.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3614912690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eacfff22-7837-4e31-a0da-5d487d903923"}]: dispatch 2024-03-19T01:46:12.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:46:12.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3614912690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eacfff22-7837-4e31-a0da-5d487d903923"}]': finished 2024-03-19T01:46:12.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: osdmap e22: 4 total, 3 up, 4 in 2024-03-19T01:46:12.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:12.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:46:12.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:46:12.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:46:12.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:46:12.260 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:46:12.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:46:12.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:46:12.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:46:12.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-19T01:46:12.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:12.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:46:12.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:46:12.261 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:46:12.262 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:46:12.262 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:46:12.262 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3614912690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eacfff22-7837-4e31-a0da-5d487d903923"}]: dispatch 2024-03-19T01:46:12.262 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:46:12.262 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3614912690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eacfff22-7837-4e31-a0da-5d487d903923"}]': finished 2024-03-19T01:46:12.262 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: osdmap e22: 4 total, 3 up, 4 in 2024-03-19T01:46:12.262 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:12.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 sudo[75927]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-19T01:46:12.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 sudo[75927]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-19T01:46:12.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 sudo[75927]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-19T01:46:12.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 sudo[75927]: pam_unix(sudo:session): session closed for user root 2024-03-19T01:46:12.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:46:12.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:46:12.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:46:12.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:46:12.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:46:12.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:46:12.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:46:12.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:46:12.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-19T01:46:12.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:12.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:46:12.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-19T01:46:12.349 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-19T01:46:12.350 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-19T01:46:12.350 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-19T01:46:12.350 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3614912690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eacfff22-7837-4e31-a0da-5d487d903923"}]: dispatch 2024-03-19T01:46:12.350 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-19T01:46:12.350 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3614912690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eacfff22-7837-4e31-a0da-5d487d903923"}]': finished 2024-03-19T01:46:12.351 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: osdmap e22: 4 total, 3 up, 4 in 2024-03-19T01:46:12.351 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:13.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:12 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/465427297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:46:13.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:12 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/465427297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:46:13.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:12 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/465427297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:46:14.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:13 smithi159 ceph-mon[67110]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:14.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:13 smithi092 ceph-mon[62574]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:14.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:13 smithi144 ceph-mon[67199]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:16.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:16 smithi159 ceph-mon[67110]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:16 smithi092 ceph-mon[62574]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:16 smithi144 ceph-mon[67199]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:18.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:18 smithi092 ceph-mon[62574]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:18.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:18 smithi159 ceph-mon[67110]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:18.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:18 smithi144 ceph-mon[67199]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:19.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-19T01:46:19.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:19.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-19T01:46:19.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:19.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-19T01:46:19.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:20 smithi092 ceph-mon[62574]: Deploying daemon osd.3 on smithi092 2024-03-19T01:46:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:20 smithi092 ceph-mon[62574]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:20.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:20 smithi144 ceph-mon[67199]: Deploying daemon osd.3 on smithi092 2024-03-19T01:46:20.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:20 smithi144 ceph-mon[67199]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:20.885 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:20 smithi159 ceph-mon[67110]: Deploying daemon osd.3 on smithi092 2024-03-19T01:46:20.885 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:20 smithi159 ceph-mon[67110]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:22.691 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:22 smithi092 ceph-mon[62574]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:22.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:22 smithi144 ceph-mon[67199]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:22 smithi159 ceph-mon[67110]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:24.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:24 smithi092 ceph-mon[62574]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:24.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:24 smithi144 ceph-mon[67199]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:24 smithi159 ceph-mon[67110]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:26.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:25 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:26.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:25 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:26.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:25 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:26.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:25 smithi159 ceph-mon[67110]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:26.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:25 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:26.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:25 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:26.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:25 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:26.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:25 smithi092 ceph-mon[62574]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:26.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:25 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:26.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:25 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:26.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:25 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:26.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:25 smithi144 ceph-mon[67199]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:28.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:28 smithi159 ceph-mon[67110]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:28.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:28 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:28.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:28 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:28.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:28 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:28.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:28 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:28.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:28 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:28 smithi092 ceph-mon[62574]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:28 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:28 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:28 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:28.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:28 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:28.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:28 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:28 smithi144 ceph-mon[67199]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:28 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:28 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:28 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:28.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:28 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:28.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:28 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:28.895 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 3 on host 'smithi092' 2024-03-19T01:46:28.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.894+0000 7fbedcff9700 1 -- 172.21.15.92:0/2755811577 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbee0062750 con 0x7fbec80a2800 2024-03-19T01:46:28.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbec80a2800 msgr2=0x7fbec80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:28.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbec80a2800 0x7fbec80a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbed4000c00 tx=0x7fbed4005fb0 comp rx=0 tx=0).stop 2024-03-19T01:46:28.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0102370 msgr2=0x7fbee0198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:28.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0102370 0x7fbee0198540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbed000e9a0 tx=0x7fbed000ecb0 comp rx=0 tx=0).stop 2024-03-19T01:46:28.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 shutdown_connections 2024-03-19T01:46:28.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbec80a2800 0x7fbec80a4cc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:28.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbee0103f10 0x7fbee019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:28.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0102370 0x7fbee0198540 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:28.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 --2- 172.21.15.92:0/2755811577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbee0103570 0x7fbee0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:28.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 >> 172.21.15.92:0/2755811577 conn(0x7fbee00fd8f0 msgr2=0x7fbee0106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:46:28.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 shutdown_connections 2024-03-19T01:46:28.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:46:28.897+0000 7fbee5b9b700 1 -- 172.21.15.92:0/2755811577 wait complete. 2024-03-19T01:46:29.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:29 smithi092 ceph-mon[62574]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-19T01:46:29.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:29 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:29.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:29 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:29.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:29 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:29.629 DEBUG:teuthology.orchestra.run.smithi092:osd.3> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.3.service 2024-03-19T01:46:29.632 INFO:tasks.cephadm:Deploying osd.4 on smithi144 with /dev/vg_nvme/lv_3... 2024-03-19T01:46:29.632 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-19T01:46:29.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:29 smithi159 ceph-mon[67110]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-19T01:46:29.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:29 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:29.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:29 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:29.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:29 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:29.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:29 smithi144 ceph-mon[67199]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-19T01:46:29.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:29 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:29.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:29 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:29.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:29 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:29.879 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:46:30.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:30 smithi144 ceph-mon[67199]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:30.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:30 smithi144 ceph-mon[67199]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-19T01:46:30.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:30 smithi144 ceph-mon[67199]: osdmap e23: 4 total, 3 up, 4 in 2024-03-19T01:46:30.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:30 smithi144 ceph-mon[67199]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:46:30.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:30 smithi092 ceph-mon[62574]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:30.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:30 smithi092 ceph-mon[62574]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-19T01:46:30.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:30 smithi092 ceph-mon[62574]: osdmap e23: 4 total, 3 up, 4 in 2024-03-19T01:46:30.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:30 smithi092 ceph-mon[62574]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:46:30.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:30.811 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 01:46:30 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3[83857]: 2024-03-19T01:46:30.461+0000 7fec79ded700 -1 osd.3 0 waiting for initial osdmap 2024-03-19T01:46:30.811 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 01:46:30 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3[83857]: 2024-03-19T01:46:30.466+0000 7fec75be0700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:46:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:30 smithi159 ceph-mon[67110]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:30 smithi159 ceph-mon[67110]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-19T01:46:30.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:30 smithi159 ceph-mon[67110]: osdmap e23: 4 total, 3 up, 4 in 2024-03-19T01:46:30.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:30 smithi159 ceph-mon[67110]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:46:30.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:31 smithi144 ceph-mon[67199]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:46:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:31 smithi144 ceph-mon[67199]: osdmap e24: 4 total, 3 up, 4 in 2024-03-19T01:46:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:31 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:31 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:31 smithi144 ceph-mon[67199]: osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] boot 2024-03-19T01:46:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:31 smithi144 ceph-mon[67199]: osdmap e25: 4 total, 4 up, 4 in 2024-03-19T01:46:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:31 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:31 smithi159 ceph-mon[67110]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:46:31.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:31 smithi159 ceph-mon[67110]: osdmap e24: 4 total, 3 up, 4 in 2024-03-19T01:46:31.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:31 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:31 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:31 smithi159 ceph-mon[67110]: osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] boot 2024-03-19T01:46:31.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:31 smithi159 ceph-mon[67110]: osdmap e25: 4 total, 4 up, 4 in 2024-03-19T01:46:31.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:31 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:31 smithi092 ceph-mon[62574]: from='osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:46:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:31 smithi092 ceph-mon[62574]: osdmap e24: 4 total, 3 up, 4 in 2024-03-19T01:46:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:31 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:31 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:31 smithi092 ceph-mon[62574]: osd.3 [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] boot 2024-03-19T01:46:31.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:31 smithi092 ceph-mon[62574]: osdmap e25: 4 total, 4 up, 4 in 2024-03-19T01:46:31.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:31 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-19T01:46:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-19T01:46:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-19T01:46:31.890 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-19T01:46:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-19T01:46:31.890 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120423 s, 871 MB/s 2024-03-19T01:46:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-19T01:46:32.524 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-03-19T01:46:32.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:32 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:46:32.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:32 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:46:32.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:32 smithi092 ceph-mon[62574]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:32.669 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:32 smithi092 ceph-mon[62574]: osdmap e26: 4 total, 4 up, 4 in 2024-03-19T01:46:32.767 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:46:32.797 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:32 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:46:32.798 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:32 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:46:32.798 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:32 smithi144 ceph-mon[67199]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:32.798 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:32 smithi144 ceph-mon[67199]: osdmap e26: 4 total, 4 up, 4 in 2024-03-19T01:46:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:32 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:46:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:32 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:46:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:32 smithi159 ceph-mon[67110]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-19T01:46:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:32 smithi159 ceph-mon[67110]: osdmap e26: 4 total, 4 up, 4 in 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi092 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi092 to 8544M 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:33.844 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: osdmap e27: 4 total, 4 up, 4 in 2024-03-19T01:46:33.845 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:33 smithi144 ceph-mon[67199]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:34.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi092 2024-03-19T01:46:34.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:34.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:34.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:34.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:34.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi092 to 8544M 2024-03-19T01:46:34.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:34.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:34.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:34.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:34.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: osdmap e27: 4 total, 4 up, 4 in 2024-03-19T01:46:34.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:33 smithi092 ceph-mon[62574]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:34.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi092 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi092 to 8544M 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: osdmap e27: 4 total, 4 up, 4 in 2024-03-19T01:46:34.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:33 smithi159 ceph-mon[67110]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:34.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.382+0000 7f7a2e304700 1 -- 172.21.15.144:0/927866695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101560 msgr2=0x7f7a281019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:34.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.382+0000 7f7a2e304700 1 --2- 172.21.15.144:0/927866695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101560 0x7f7a281019c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009990 tx=0x7f7a1c009ca0 comp rx=0 tx=0).stop 2024-03-19T01:46:34.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.382+0000 7f7a2e304700 1 -- 172.21.15.144:0/927866695 shutdown_connections 2024-03-19T01:46:34.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.382+0000 7f7a2e304700 1 --2- 172.21.15.144:0/927866695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a28101f00 0x7f7a281097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:34.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.382+0000 7f7a2e304700 1 --2- 172.21.15.144:0/927866695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a28102820 0x7f7a28102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:34.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.382+0000 7f7a2e304700 1 --2- 172.21.15.144:0/927866695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101560 0x7f7a281019c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:34.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.382+0000 7f7a2e304700 1 -- 172.21.15.144:0/927866695 >> 172.21.15.144:0/927866695 conn(0x7f7a280fbfb0 msgr2=0x7f7a280fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:46:34.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.382+0000 7f7a2e304700 1 -- 172.21.15.144:0/927866695 shutdown_connections 2024-03-19T01:46:34.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.383+0000 7f7a2e304700 1 -- 172.21.15.144:0/927866695 wait complete. 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.383+0000 7f7a2e304700 1 Processor -- start 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.383+0000 7f7a2e304700 1 -- start start 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.383+0000 7f7a2e304700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a28101560 0x7f7a281986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.383+0000 7f7a2e304700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101f00 0x7f7a28198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.383+0000 7f7a2e304700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a28102820 0x7f7a2819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.383+0000 7f7a2e304700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2810cf70 con 0x7f7a28101560 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.383+0000 7f7a2e304700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2810cdf0 con 0x7f7a28101f00 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.383+0000 7f7a2e304700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2810d0f0 con 0x7f7a28102820 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a277fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101f00 0x7f7a28198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a277fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101f00 0x7f7a28198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55058/0 (socket says 172.21.15.144:55058) 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a277fe700 1 -- 172.21.15.144:0/535029608 learned_addr learned my addr 172.21.15.144:0/535029608 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a2c8a1700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a28102820 0x7f7a2819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a27fff700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a28101560 0x7f7a281986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:34.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a277fe700 1 -- 172.21.15.144:0/535029608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a28102820 msgr2=0x7f7a2819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a277fe700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a28102820 0x7f7a2819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a277fe700 1 -- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a28101560 msgr2=0x7f7a281986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a277fe700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a28101560 0x7f7a281986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a277fe700 1 -- 172.21.15.144:0/535029608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a281a0300 con 0x7f7a28101f00 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a277fe700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101f00 0x7f7a28198be0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c005910 tx=0x7f7a1c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a257fa700 1 -- 172.21.15.144:0/535029608 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c027070 con 0x7f7a28101f00 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a281a0590 con 0x7f7a28101f00 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a257fa700 1 -- 172.21.15.144:0/535029608 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1c022470 con 0x7f7a28101f00 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a257fa700 1 -- 172.21.15.144:0/535029608 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00f6a0 con 0x7f7a28101f00 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.384+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a281a0ac0 con 0x7f7a28101f00 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.385+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a28068fd0 con 0x7f7a28101f00 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.386+0000 7f7a257fa700 1 -- 172.21.15.144:0/535029608 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7a1c02a070 con 0x7f7a28101f00 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.387+0000 7f7a257fa700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7a100a2890 0x7f7a100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:34.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.387+0000 7f7a257fa700 1 -- 172.21.15.144:0/535029608 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f7a1c0cea60 con 0x7f7a28101f00 2024-03-19T01:46:34.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.387+0000 7f7a27fff700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7a100a2890 0x7f7a100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:34.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.388+0000 7f7a27fff700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7a100a2890 0x7f7a100a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a18006fd0 tx=0x7f7a18008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:46:34.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.391+0000 7f7a257fa700 1 -- 172.21.15.144:0/535029608 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7a1c013070 con 0x7f7a28101f00 2024-03-19T01:46:34.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:34.596+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7a281a0de0 con 0x7f7a100a2890 2024-03-19T01:46:35.190 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:34 smithi144 ceph-mon[67199]: osdmap e28: 4 total, 4 up, 4 in 2024-03-19T01:46:35.190 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:34 smithi144 ceph-mon[67199]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:46:35.190 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:46:35.190 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:46:35.190 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:35.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:34 smithi159 ceph-mon[67110]: osdmap e28: 4 total, 4 up, 4 in 2024-03-19T01:46:35.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:34 smithi159 ceph-mon[67110]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:46:35.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:46:35.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:46:35.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:35.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:34 smithi092 ceph-mon[62574]: osdmap e28: 4 total, 4 up, 4 in 2024-03-19T01:46:35.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:34 smithi092 ceph-mon[62574]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:46:35.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:46:35.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:46:35.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:36.192 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:35 smithi144 ceph-mon[67199]: osdmap e29: 4 total, 4 up, 4 in 2024-03-19T01:46:36.192 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:35 smithi144 ceph-mon[67199]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-19T01:46:36.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:35 smithi159 ceph-mon[67110]: osdmap e29: 4 total, 4 up, 4 in 2024-03-19T01:46:36.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:35 smithi159 ceph-mon[67110]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-19T01:46:36.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:35 smithi092 ceph-mon[62574]: osdmap e29: 4 total, 4 up, 4 in 2024-03-19T01:46:36.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:35 smithi092 ceph-mon[62574]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-19T01:46:37.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:36 smithi159 ceph-mon[67110]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-19T01:46:37.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:36 smithi092 ceph-mon[62574]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-19T01:46:37.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:36 smithi144 ceph-mon[67199]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-19T01:46:38.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:37 smithi144 ceph-mon[67199]: from='client.? 172.21.15.144:0/2653459795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f152f6-a121-47d6-8169-0381921ac5ee"}]: dispatch 2024-03-19T01:46:38.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:37 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f152f6-a121-47d6-8169-0381921ac5ee"}]: dispatch 2024-03-19T01:46:38.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:37 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1f152f6-a121-47d6-8169-0381921ac5ee"}]': finished 2024-03-19T01:46:38.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:37 smithi144 ceph-mon[67199]: osdmap e30: 5 total, 4 up, 5 in 2024-03-19T01:46:38.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:37 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:38.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:37 smithi144 ceph-mon[67199]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-19T01:46:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:37 smithi159 ceph-mon[67110]: from='client.? 172.21.15.144:0/2653459795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f152f6-a121-47d6-8169-0381921ac5ee"}]: dispatch 2024-03-19T01:46:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:37 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f152f6-a121-47d6-8169-0381921ac5ee"}]: dispatch 2024-03-19T01:46:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:37 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1f152f6-a121-47d6-8169-0381921ac5ee"}]': finished 2024-03-19T01:46:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:37 smithi159 ceph-mon[67110]: osdmap e30: 5 total, 4 up, 5 in 2024-03-19T01:46:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:37 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:38.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:37 smithi159 ceph-mon[67110]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-19T01:46:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:37 smithi092 ceph-mon[62574]: from='client.? 172.21.15.144:0/2653459795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f152f6-a121-47d6-8169-0381921ac5ee"}]: dispatch 2024-03-19T01:46:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:37 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f152f6-a121-47d6-8169-0381921ac5ee"}]: dispatch 2024-03-19T01:46:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:37 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1f152f6-a121-47d6-8169-0381921ac5ee"}]': finished 2024-03-19T01:46:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:37 smithi092 ceph-mon[62574]: osdmap e30: 5 total, 4 up, 5 in 2024-03-19T01:46:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:37 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:37 smithi092 ceph-mon[62574]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-19T01:46:39.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:38 smithi159 ceph-mon[67110]: from='client.? 172.21.15.144:0/2294310163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:46:39.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:38 smithi092 ceph-mon[62574]: from='client.? 172.21.15.144:0/2294310163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:46:39.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:38 smithi144 ceph-mon[67199]: from='client.? 172.21.15.144:0/2294310163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:46:40.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:39 smithi159 ceph-mon[67110]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T01:46:40.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:39 smithi092 ceph-mon[62574]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T01:46:40.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:39 smithi144 ceph-mon[67199]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T01:46:41.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:40 smithi159 ceph-mon[67110]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-19T01:46:41.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:40 smithi159 ceph-mon[67110]: Cluster is now healthy 2024-03-19T01:46:41.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:40 smithi092 ceph-mon[62574]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-19T01:46:41.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:40 smithi092 ceph-mon[62574]: Cluster is now healthy 2024-03-19T01:46:41.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:40 smithi144 ceph-mon[67199]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-19T01:46:41.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:40 smithi144 ceph-mon[67199]: Cluster is now healthy 2024-03-19T01:46:42.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:41 smithi144 ceph-mon[67199]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-19T01:46:42.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:41 smithi159 ceph-mon[67110]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-19T01:46:42.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:41 smithi092 ceph-mon[62574]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-19T01:46:44.693 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:44 smithi144 ceph-mon[67199]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-19T01:46:44.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:44 smithi159 ceph-mon[67110]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-19T01:46:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:44 smithi092 ceph-mon[62574]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-19T01:46:45.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-19T01:46:45.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:45.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-19T01:46:45.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:45.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-19T01:46:45.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:46.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:46 smithi144 ceph-mon[67199]: Deploying daemon osd.4 on smithi144 2024-03-19T01:46:46.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:46 smithi144 ceph-mon[67199]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:46:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:46 smithi092 ceph-mon[62574]: Deploying daemon osd.4 on smithi144 2024-03-19T01:46:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:46 smithi092 ceph-mon[62574]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:46:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:46 smithi159 ceph-mon[67110]: Deploying daemon osd.4 on smithi144 2024-03-19T01:46:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:46 smithi159 ceph-mon[67110]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:46:48.769 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:48 smithi144 ceph-mon[67199]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-19T01:46:48.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:48 smithi092 ceph-mon[62574]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-19T01:46:48.957 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:48 smithi159 ceph-mon[67110]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-19T01:46:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:50 smithi144 ceph-mon[67199]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-19T01:46:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:50.598 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:50.757 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:50 smithi159 ceph-mon[67110]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-19T01:46:50.757 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:50.758 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:50.758 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:50 smithi092 ceph-mon[62574]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-19T01:46:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:52.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:52 smithi144 ceph-mon[67199]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:52 smithi092 ceph-mon[62574]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:52 smithi159 ceph-mon[67110]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:54.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.211+0000 7f7a257fa700 1 -- 172.21.15.144:0/535029608 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7a281a0de0 con 0x7f7a100a2890 2024-03-19T01:46:54.212 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 4 on host 'smithi144' 2024-03-19T01:46:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7a100a2890 msgr2=0x7f7a100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7a100a2890 0x7f7a100a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a18006fd0 tx=0x7f7a18008040 comp rx=0 tx=0).stop 2024-03-19T01:46:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101f00 msgr2=0x7f7a28198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101f00 0x7f7a28198be0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c005910 tx=0x7f7a1c0047b0 comp rx=0 tx=0).stop 2024-03-19T01:46:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 shutdown_connections 2024-03-19T01:46:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7a100a2890 0x7f7a100a4d50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a28102820 0x7f7a2819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a28101560 0x7f7a281986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 --2- 172.21.15.144:0/535029608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a28101f00 0x7f7a28198be0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 >> 172.21.15.144:0/535029608 conn(0x7f7a280fbfb0 msgr2=0x7f7a280fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:46:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 shutdown_connections 2024-03-19T01:46:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:46:54.214+0000 7f7a2e304700 1 -- 172.21.15.144:0/535029608 wait complete. 2024-03-19T01:46:54.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:54.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:54.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:54 smithi144 ceph-mon[67199]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:54.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.468 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:54.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:54.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:54.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:54 smithi092 ceph-mon[62574]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:54.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:54.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:54 smithi159 ceph-mon[67110]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:54.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:46:54.815 DEBUG:teuthology.orchestra.run.smithi144:osd.4> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.4.service 2024-03-19T01:46:54.819 INFO:tasks.cephadm:Deploying osd.5 on smithi159 with /dev/vg_nvme/lv_3... 2024-03-19T01:46:54.819 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-19T01:46:55.065 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:46:55.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:55 smithi159 ceph-mon[67110]: from='osd.4 [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:46:55.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:55 smithi159 ceph-mon[67110]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:46:55.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:55 smithi092 ceph-mon[62574]: from='osd.4 [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:46:55.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:55 smithi092 ceph-mon[62574]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:46:55.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:55 smithi144 ceph-mon[67199]: from='osd.4 [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:46:55.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:55 smithi144 ceph-mon[67199]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-19T01:46:56.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:56 smithi159 ceph-mon[67110]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-19T01:46:56.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:56 smithi159 ceph-mon[67110]: osdmap e31: 5 total, 4 up, 5 in 2024-03-19T01:46:56.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:56 smithi159 ceph-mon[67110]: from='osd.4 [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:46:56.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:56.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:56 smithi159 ceph-mon[67110]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:46:56.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:56 smithi159 ceph-mon[67110]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:56.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:56 smithi092 ceph-mon[62574]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-19T01:46:56.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:56 smithi092 ceph-mon[62574]: osdmap e31: 5 total, 4 up, 5 in 2024-03-19T01:46:56.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:56 smithi092 ceph-mon[62574]: from='osd.4 [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:46:56.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:56.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:56 smithi092 ceph-mon[62574]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:46:56.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:56 smithi092 ceph-mon[62574]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:56.596 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 01:46:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4[78733]: 2024-03-19T01:46:56.252+0000 7f0fa26b8700 -1 osd.4 0 waiting for initial osdmap 2024-03-19T01:46:56.596 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 01:46:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4[78733]: 2024-03-19T01:46:56.257+0000 7f0f9e4ab700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:46:56.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:56 smithi144 ceph-mon[67199]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-19T01:46:56.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:56 smithi144 ceph-mon[67199]: osdmap e31: 5 total, 4 up, 5 in 2024-03-19T01:46:56.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:56 smithi144 ceph-mon[67199]: from='osd.4 [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:46:56.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:56.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:56 smithi144 ceph-mon[67199]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:46:56.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:56 smithi144 ceph-mon[67199]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:57.035 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-19T01:46:57.035 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-19T01:46:57.035 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-19T01:46:57.035 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-19T01:46:57.035 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112765 s, 930 MB/s 2024-03-19T01:46:57.035 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-19T01:46:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:57 smithi144 ceph-mon[67199]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:46:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:57 smithi144 ceph-mon[67199]: osdmap e32: 5 total, 4 up, 5 in 2024-03-19T01:46:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:57 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:57 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:57.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:57 smithi092 ceph-mon[62574]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:46:57.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:57 smithi092 ceph-mon[62574]: osdmap e32: 5 total, 4 up, 5 in 2024-03-19T01:46:57.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:57 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:57.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:57 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:57.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:57 smithi159 ceph-mon[67110]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:46:57.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:57 smithi159 ceph-mon[67110]: osdmap e32: 5 total, 4 up, 5 in 2024-03-19T01:46:57.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:57 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:57.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:57 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:57.825 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2024-03-19T01:46:58.061 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:46:58.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: osd.4 [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] boot 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: osdmap e33: 5 total, 5 up, 5 in 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:58.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:46:58.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:46:58.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: osd.4 [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] boot 2024-03-19T01:46:58.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: osdmap e33: 5 total, 5 up, 5 in 2024-03-19T01:46:58.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:58.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:58.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:58.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:58.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:58.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:58.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:46:58.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:46:58.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: osd.4 [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] boot 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: osdmap e33: 5 total, 5 up, 5 in 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:46:58.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:46:59.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:59 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi144 2024-03-19T01:46:59.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:59 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi144 to 8540M 2024-03-19T01:46:59.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:46:59 smithi159 ceph-mon[67110]: osdmap e34: 5 total, 5 up, 5 in 2024-03-19T01:46:59.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:59 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi144 2024-03-19T01:46:59.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:59 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi144 to 8540M 2024-03-19T01:46:59.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:46:59 smithi092 ceph-mon[62574]: osdmap e34: 5 total, 5 up, 5 in 2024-03-19T01:46:59.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:59 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi144 2024-03-19T01:46:59.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:59 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi144 to 8540M 2024-03-19T01:46:59.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:46:59 smithi144 ceph-mon[67199]: osdmap e34: 5 total, 5 up, 5 in 2024-03-19T01:46:59.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 -- 172.21.15.159:0/3041293319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30103f00 msgr2=0x7fad3010a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:59.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 --2- 172.21.15.159:0/3041293319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30103f00 0x7fad3010a7b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fad2800b0a0 tx=0x7fad2800b3b0 comp rx=0 tx=0).stop 2024-03-19T01:46:59.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 -- 172.21.15.159:0/3041293319 shutdown_connections 2024-03-19T01:46:59.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 --2- 172.21.15.159:0/3041293319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30103f00 0x7fad3010a7b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:59.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 --2- 172.21.15.159:0/3041293319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad30102360 0x7fad30102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:59.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 --2- 172.21.15.159:0/3041293319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad30103560 0x7fad301039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 -- 172.21.15.159:0/3041293319 >> 172.21.15.159:0/3041293319 conn(0x7fad300fd920 msgr2=0x7fad300ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 -- 172.21.15.159:0/3041293319 shutdown_connections 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 -- 172.21.15.159:0/3041293319 wait complete. 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 Processor -- start 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 -- start start 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30102360 0x7fad301985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad30103560 0x7fad30198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad30103f00 0x7fad3019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad3010e010 con 0x7fad30103560 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad3010de90 con 0x7fad30103f00 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.766+0000 7fad36daf700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad3010e190 con 0x7fad30102360 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad34b4b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30102360 0x7fad301985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:59.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad34b4b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30102360 0x7fad301985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43466/0 (socket says 172.21.15.159:43466) 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad34b4b700 1 -- 172.21.15.159:0/3514450949 learned_addr learned my addr 172.21.15.159:0/3514450949 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad3534c700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad30103f00 0x7fad3019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad2ffff700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad30103560 0x7fad30198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad34b4b700 1 -- 172.21.15.159:0/3514450949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad30103f00 msgr2=0x7fad3019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad34b4b700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad30103f00 0x7fad3019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad34b4b700 1 -- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad30103560 msgr2=0x7fad30198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad34b4b700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad30103560 0x7fad30198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad34b4b700 1 -- 172.21.15.159:0/3514450949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad301a0200 con 0x7fad30102360 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad34b4b700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30102360 0x7fad301985a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fad2400d9b0 tx=0x7fad2400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad2dffb700 1 -- 172.21.15.159:0/3514450949 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24017070 con 0x7fad30102360 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad2dffb700 1 -- 172.21.15.159:0/3514450949 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad24009e10 con 0x7fad30102360 2024-03-19T01:46:59.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad301a04f0 con 0x7fad30102360 2024-03-19T01:46:59.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.767+0000 7fad2dffb700 1 -- 172.21.15.159:0/3514450949 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2400f620 con 0x7fad30102360 2024-03-19T01:46:59.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.768+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad301a0ab0 con 0x7fad30102360 2024-03-19T01:46:59.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.769+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad30068fd0 con 0x7fad30102360 2024-03-19T01:46:59.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.770+0000 7fad2dffb700 1 -- 172.21.15.159:0/3514450949 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fad24009930 con 0x7fad30102360 2024-03-19T01:46:59.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.770+0000 7fad2dffb700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad200a2910 0x7fad200a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:46:59.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.770+0000 7fad2dffb700 1 -- 172.21.15.159:0/3514450949 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fad240cd510 con 0x7fad30102360 2024-03-19T01:46:59.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.770+0000 7fad2ffff700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad200a2910 0x7fad200a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:46:59.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.771+0000 7fad2ffff700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad200a2910 0x7fad200a4dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad1c006fd0 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:46:59.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.775+0000 7fad2dffb700 1 -- 172.21.15.159:0/3514450949 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fad2409b620 con 0x7fad30102360 2024-03-19T01:46:59.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:46:59.980+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fad30062750 con 0x7fad200a2910 2024-03-19T01:47:00.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:00 smithi092 ceph-mon[62574]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:00.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:47:00.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:47:00.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:00.582 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:00 smithi159 ceph-mon[67110]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:00.583 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:47:00.583 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:47:00.583 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:00.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:00 smithi144 ceph-mon[67199]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:00.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:47:00.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:47:00.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:01.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:01 smithi092 ceph-mon[62574]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:47:01.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:01 smithi144 ceph-mon[67199]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:47:01.605 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:01 smithi159 ceph-mon[67110]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:47:02.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:02 smithi159 ceph-mon[67110]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:02.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:02 smithi092 ceph-mon[62574]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:02.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:02 smithi144 ceph-mon[67199]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:03.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:03 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/1292198044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e86f624-5f0c-4fc4-a12d-1d47750a4428"}]: dispatch 2024-03-19T01:47:03.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:03 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e86f624-5f0c-4fc4-a12d-1d47750a4428"}]: dispatch 2024-03-19T01:47:03.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:03 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e86f624-5f0c-4fc4-a12d-1d47750a4428"}]': finished 2024-03-19T01:47:03.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:03 smithi092 ceph-mon[62574]: osdmap e35: 6 total, 5 up, 6 in 2024-03-19T01:47:03.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:03.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:03 smithi144 ceph-mon[67199]: from='client.? 172.21.15.159:0/1292198044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e86f624-5f0c-4fc4-a12d-1d47750a4428"}]: dispatch 2024-03-19T01:47:03.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:03 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e86f624-5f0c-4fc4-a12d-1d47750a4428"}]: dispatch 2024-03-19T01:47:03.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:03 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e86f624-5f0c-4fc4-a12d-1d47750a4428"}]': finished 2024-03-19T01:47:03.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:03 smithi144 ceph-mon[67199]: osdmap e35: 6 total, 5 up, 6 in 2024-03-19T01:47:03.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:03 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:03.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:03 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/1292198044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e86f624-5f0c-4fc4-a12d-1d47750a4428"}]: dispatch 2024-03-19T01:47:03.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:03 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e86f624-5f0c-4fc4-a12d-1d47750a4428"}]: dispatch 2024-03-19T01:47:03.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:03 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e86f624-5f0c-4fc4-a12d-1d47750a4428"}]': finished 2024-03-19T01:47:03.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:03 smithi159 ceph-mon[67110]: osdmap e35: 6 total, 5 up, 6 in 2024-03-19T01:47:03.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:03 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:04.534 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:04 smithi144 ceph-mon[67199]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:04.534 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:04 smithi144 ceph-mon[67199]: from='client.? 172.21.15.159:0/2687078278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:47:04.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:04 smithi092 ceph-mon[62574]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:04.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:04 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/2687078278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:47:04.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:04 smithi159 ceph-mon[67110]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:04.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:04 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/2687078278' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:47:06.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:06 smithi159 ceph-mon[67110]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:06.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:06 smithi092 ceph-mon[62574]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:06 smithi144 ceph-mon[67199]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:08.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:08 smithi159 ceph-mon[67110]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:08 smithi092 ceph-mon[62574]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:08.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:08 smithi144 ceph-mon[67199]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:10.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:10 smithi159 ceph-mon[67110]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:10.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-19T01:47:10.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:10.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:10 smithi092 ceph-mon[62574]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:10.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-19T01:47:10.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:10.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:10 smithi144 ceph-mon[67199]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:10.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-19T01:47:10.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:11.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:11 smithi159 ceph-mon[67110]: Deploying daemon osd.5 on smithi159 2024-03-19T01:47:11.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:11 smithi092 ceph-mon[62574]: Deploying daemon osd.5 on smithi159 2024-03-19T01:47:11.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:11 smithi144 ceph-mon[67199]: Deploying daemon osd.5 on smithi159 2024-03-19T01:47:12.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:12 smithi092 ceph-mon[62574]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:12 smithi144 ceph-mon[67199]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:12.926 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:12 smithi159 ceph-mon[67110]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:14.682 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:14 smithi159 ceph-mon[67110]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:14.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:14 smithi092 ceph-mon[62574]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:14 smithi144 ceph-mon[67199]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:16.614 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:16 smithi159 ceph-mon[67110]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:16.614 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:16.614 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:16.614 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:16 smithi092 ceph-mon[62574]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:16 smithi144 ceph-mon[67199]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:18.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:18 smithi159 ceph-mon[67110]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:18.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:18 smithi092 ceph-mon[62574]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:18.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:18 smithi144 ceph-mon[67199]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:19.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.093+0000 7fad2dffb700 1 -- 172.21.15.159:0/3514450949 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fad30062750 con 0x7fad200a2910 2024-03-19T01:47:19.094 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 5 on host 'smithi159' 2024-03-19T01:47:19.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad200a2910 msgr2=0x7fad200a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:19.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad200a2910 0x7fad200a4dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad1c006fd0 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2024-03-19T01:47:19.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30102360 msgr2=0x7fad301985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30102360 0x7fad301985a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fad2400d9b0 tx=0x7fad2400dcc0 comp rx=0 tx=0).stop 2024-03-19T01:47:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 shutdown_connections 2024-03-19T01:47:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad200a2910 0x7fad200a4dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad30103f00 0x7fad3019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad30102360 0x7fad301985a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 --2- 172.21.15.159:0/3514450949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad30103560 0x7fad30198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 >> 172.21.15.159:0/3514450949 conn(0x7fad300fd920 msgr2=0x7fad30106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:47:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 shutdown_connections 2024-03-19T01:47:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:47:19.096+0000 7fad36daf700 1 -- 172.21.15.159:0/3514450949 wait complete. 2024-03-19T01:47:20.115 DEBUG:teuthology.orchestra.run.smithi159:osd.5> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.5.service 2024-03-19T01:47:20.118 INFO:tasks.cephadm:Deploying osd.6 on smithi092 with /dev/vg_nvme/lv_2... 2024-03-19T01:47:20.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-19T01:47:20.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:20.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:20.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:20 smithi144 ceph-mon[67199]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:20.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:20 smithi144 ceph-mon[67199]: from='osd.5 [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:47:20.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:20 smithi144 ceph-mon[67199]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:47:20.382 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:47:20.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:20.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:20.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:20 smithi159 ceph-mon[67110]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:20.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:20 smithi159 ceph-mon[67110]: from='osd.5 [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:47:20.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:20 smithi159 ceph-mon[67110]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:47:20.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:20.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:20.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:20 smithi092 ceph-mon[62574]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:20.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:20 smithi092 ceph-mon[62574]: from='osd.5 [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:47:20.504 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:20 smithi092 ceph-mon[62574]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-19T01:47:21.132 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:21 smithi159 ceph-mon[67110]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-19T01:47:21.133 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:21 smithi159 ceph-mon[67110]: osdmap e36: 6 total, 5 up, 6 in 2024-03-19T01:47:21.133 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:21 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:21.133 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:21 smithi159 ceph-mon[67110]: from='osd.5 [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:47:21.133 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:21 smithi159 ceph-mon[67110]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:47:21.133 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:21 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:21.133 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:21 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:21.133 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 01:47:21 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T01:47:21.099+0000 7fce8c6ca700 -1 osd.5 0 waiting for initial osdmap 2024-03-19T01:47:21.133 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 01:47:21 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T01:47:21.105+0000 7fce884bd700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:47:21.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:21 smithi144 ceph-mon[67199]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-19T01:47:21.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:21 smithi144 ceph-mon[67199]: osdmap e36: 6 total, 5 up, 6 in 2024-03-19T01:47:21.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:21 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:21.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:21 smithi144 ceph-mon[67199]: from='osd.5 [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:47:21.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:21 smithi144 ceph-mon[67199]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:47:21.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:21 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:21.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:21 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:21.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:21 smithi092 ceph-mon[62574]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-19T01:47:21.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:21 smithi092 ceph-mon[62574]: osdmap e36: 6 total, 5 up, 6 in 2024-03-19T01:47:21.541 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:21 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:21.542 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:21 smithi092 ceph-mon[62574]: from='osd.5 [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:47:21.542 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:21 smithi092 ceph-mon[62574]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:47:21.542 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:21 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:21.542 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:21 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:22.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:22 smithi159 ceph-mon[67110]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:47:22.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:22 smithi159 ceph-mon[67110]: osdmap e37: 6 total, 5 up, 6 in 2024-03-19T01:47:22.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:22 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:22.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:22 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:22.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:22 smithi159 ceph-mon[67110]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:22.481 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-19T01:47:22.481 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-19T01:47:22.481 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-03-19T01:47:22.481 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-03-19T01:47:22.481 INFO:teuthology.orchestra.run.smithi092.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113392 s, 925 MB/s 2024-03-19T01:47:22.482 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-03-19T01:47:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:22 smithi092 ceph-mon[62574]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:47:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:22 smithi092 ceph-mon[62574]: osdmap e37: 6 total, 5 up, 6 in 2024-03-19T01:47:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:22.482 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:22 smithi092 ceph-mon[62574]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:22.526 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:22 smithi144 ceph-mon[67199]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:47:22.526 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:22 smithi144 ceph-mon[67199]: osdmap e37: 6 total, 5 up, 6 in 2024-03-19T01:47:22.526 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:22 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:22.526 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:22 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:22.526 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:22 smithi144 ceph-mon[67199]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-19T01:47:23.135 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:23 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:47:23.135 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:23 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:47:23.136 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:23.136 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:23 smithi092 ceph-mon[62574]: osd.5 [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] boot 2024-03-19T01:47:23.136 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:23 smithi092 ceph-mon[62574]: osdmap e38: 6 total, 6 up, 6 in 2024-03-19T01:47:23.136 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:23.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-03-19T01:47:23.386 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:23 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:47:23.386 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:23 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:47:23.386 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:23.386 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:23 smithi159 ceph-mon[67110]: osd.5 [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] boot 2024-03-19T01:47:23.386 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:23 smithi159 ceph-mon[67110]: osdmap e38: 6 total, 6 up, 6 in 2024-03-19T01:47:23.386 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:47:23.450 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:23 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:47:23.450 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:23 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:47:23.450 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:23.450 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:23 smithi144 ceph-mon[67199]: osd.5 [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] boot 2024-03-19T01:47:23.450 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:23 smithi144 ceph-mon[67199]: osdmap e38: 6 total, 6 up, 6 in 2024-03-19T01:47:23.450 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-19T01:47:24.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: osdmap e39: 6 total, 6 up, 6 in 2024-03-19T01:47:24.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:24.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:24.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: osdmap e39: 6 total, 6 up, 6 in 2024-03-19T01:47:24.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:24.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: osdmap e39: 6 total, 6 up, 6 in 2024-03-19T01:47:24.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:24.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.201+0000 7fe0567c2700 1 -- 172.21.15.92:0/3277811937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0501036c0 msgr2=0x7fe050103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.201+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3277811937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0501036c0 0x7fe050103b20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe0440099c0 tx=0x7fe044009cd0 comp rx=0 tx=0).stop 2024-03-19T01:47:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.201+0000 7fe0567c2700 1 -- 172.21.15.92:0/3277811937 shutdown_connections 2024-03-19T01:47:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.201+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3277811937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe050104060 0x7fe05010a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.201+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3277811937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050100640 0x7fe050100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.201+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3277811937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0501036c0 0x7fe050103b20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.201+0000 7fe0567c2700 1 -- 172.21.15.92:0/3277811937 >> 172.21.15.92:0/3277811937 conn(0x7fe0500fbfa0 msgr2=0x7fe0500fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:47:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.201+0000 7fe0567c2700 1 -- 172.21.15.92:0/3277811937 shutdown_connections 2024-03-19T01:47:25.203 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.201+0000 7fe0567c2700 1 -- 172.21.15.92:0/3277811937 wait complete. 2024-03-19T01:47:25.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:25 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi159 2024-03-19T01:47:25.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:25 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi159 to 8540M 2024-03-19T01:47:25.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:25 smithi092 ceph-mon[62574]: osdmap e40: 6 total, 6 up, 6 in 2024-03-19T01:47:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.202+0000 7fe0567c2700 1 Processor -- start 2024-03-19T01:47:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.202+0000 7fe0567c2700 1 -- start start 2024-03-19T01:47:25.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.202+0000 7fe0567c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050100640 0x7fe050198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:47:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.202+0000 7fe0567c2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501036c0 0x7fe050198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:47:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.202+0000 7fe0567c2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe050104060 0x7fe05019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:47:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050100640 0x7fe050198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:47:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe0567c2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe05010dfc0 con 0x7fe050100640 2024-03-19T01:47:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe0567c2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe05010de40 con 0x7fe0501036c0 2024-03-19T01:47:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe0567c2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe05010e140 con 0x7fe050104060 2024-03-19T01:47:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050100640 0x7fe050198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35562/0 (socket says 172.21.15.92:35562) 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04ffff700 1 -- 172.21.15.92:0/3617107839 learned_addr learned my addr 172.21.15.92:0/3617107839 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe054d5f700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe050104060 0x7fe05019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04f7fe700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501036c0 0x7fe050198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04ffff700 1 -- 172.21.15.92:0/3617107839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe050104060 msgr2=0x7fe05019fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04ffff700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe050104060 0x7fe05019fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04ffff700 1 -- 172.21.15.92:0/3617107839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501036c0 msgr2=0x7fe050198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04ffff700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501036c0 0x7fe050198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04ffff700 1 -- 172.21.15.92:0/3617107839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0501a0170 con 0x7fe050100640 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04f7fe700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501036c0 0x7fe050198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:47:25.207 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04ffff700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050100640 0x7fe050198510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe04000b530 tx=0x7fe04000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:47:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04d7fa700 1 -- 172.21.15.92:0/3617107839 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040015020 con 0x7fe050100640 2024-03-19T01:47:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04d7fa700 1 -- 172.21.15.92:0/3617107839 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe040011d10 con 0x7fe050100640 2024-03-19T01:47:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0501a0460 con 0x7fe050100640 2024-03-19T01:47:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.203+0000 7fe04d7fa700 1 -- 172.21.15.92:0/3617107839 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04000f2e0 con 0x7fe050100640 2024-03-19T01:47:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.204+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0501a0a70 con 0x7fe050100640 2024-03-19T01:47:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.205+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe050068fd0 con 0x7fe050100640 2024-03-19T01:47:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.206+0000 7fe04d7fa700 1 -- 172.21.15.92:0/3617107839 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe040018020 con 0x7fe050100640 2024-03-19T01:47:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.206+0000 7fe04d7fa700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0380a2910 0x7fe0380a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:47:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.206+0000 7fe04f7fe700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0380a2910 0x7fe0380a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:47:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.206+0000 7fe04d7fa700 1 -- 172.21.15.92:0/3617107839 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4533+0+0 (secure 0 0 0) 0x7fe0400cd240 con 0x7fe050100640 2024-03-19T01:47:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.207+0000 7fe04f7fe700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0380a2910 0x7fe0380a4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe044000c00 tx=0x7fe044003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:47:25.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.211+0000 7fe04d7fa700 1 -- 172.21.15.92:0/3617107839 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe04001f020 con 0x7fe050100640 2024-03-19T01:47:25.419 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:25.418+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe050062750 con 0x7fe0380a2910 2024-03-19T01:47:25.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:25 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi159 2024-03-19T01:47:25.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:25 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi159 to 8540M 2024-03-19T01:47:25.458 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:25 smithi159 ceph-mon[67110]: osdmap e40: 6 total, 6 up, 6 in 2024-03-19T01:47:25.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:25 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi159 2024-03-19T01:47:25.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:25 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi159 to 8540M 2024-03-19T01:47:25.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:25 smithi144 ceph-mon[67199]: osdmap e40: 6 total, 6 up, 6 in 2024-03-19T01:47:26.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:26 smithi092 ceph-mon[62574]: osdmap e41: 6 total, 6 up, 6 in 2024-03-19T01:47:26.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:26 smithi092 ceph-mon[62574]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:26.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:26 smithi092 ceph-mon[62574]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:47:26.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:47:26.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:47:26.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:26.270 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:26 smithi092 ceph-mon[62574]: osdmap e42: 6 total, 6 up, 6 in 2024-03-19T01:47:26.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:26 smithi159 ceph-mon[67110]: osdmap e41: 6 total, 6 up, 6 in 2024-03-19T01:47:26.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:26 smithi159 ceph-mon[67110]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:26.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:26 smithi159 ceph-mon[67110]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:47:26.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:47:26.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:47:26.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:26.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:26 smithi159 ceph-mon[67110]: osdmap e42: 6 total, 6 up, 6 in 2024-03-19T01:47:26.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:26 smithi144 ceph-mon[67199]: osdmap e41: 6 total, 6 up, 6 in 2024-03-19T01:47:26.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:26 smithi144 ceph-mon[67199]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:26.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:26 smithi144 ceph-mon[67199]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:47:26.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:47:26.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:47:26.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:26.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:26 smithi144 ceph-mon[67199]: osdmap e42: 6 total, 6 up, 6 in 2024-03-19T01:47:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:28 smithi092 ceph-mon[62574]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:28 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/358353646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8595632-012c-480c-9324-994266f9a4d2"}]: dispatch 2024-03-19T01:47:28.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:28 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/358353646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8595632-012c-480c-9324-994266f9a4d2"}]': finished 2024-03-19T01:47:28.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:28 smithi092 ceph-mon[62574]: osdmap e43: 7 total, 6 up, 7 in 2024-03-19T01:47:28.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:28 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:28 smithi144 ceph-mon[67199]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:28 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/358353646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8595632-012c-480c-9324-994266f9a4d2"}]: dispatch 2024-03-19T01:47:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:28 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/358353646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8595632-012c-480c-9324-994266f9a4d2"}]': finished 2024-03-19T01:47:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:28 smithi144 ceph-mon[67199]: osdmap e43: 7 total, 6 up, 7 in 2024-03-19T01:47:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:28 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:28 smithi159 ceph-mon[67110]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:28 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/358353646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8595632-012c-480c-9324-994266f9a4d2"}]: dispatch 2024-03-19T01:47:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:28 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/358353646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8595632-012c-480c-9324-994266f9a4d2"}]': finished 2024-03-19T01:47:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:28 smithi159 ceph-mon[67110]: osdmap e43: 7 total, 6 up, 7 in 2024-03-19T01:47:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:28 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:29.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:29 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3292397575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:47:29.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:29 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3292397575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:47:29.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:29 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3292397575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:47:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:30 smithi092 ceph-mon[62574]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-19T01:47:30.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:30 smithi144 ceph-mon[67199]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-19T01:47:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:30 smithi159 ceph-mon[67110]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-19T01:47:32.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:32 smithi159 ceph-mon[67110]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-19T01:47:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:32 smithi092 ceph-mon[62574]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-19T01:47:32.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:32 smithi144 ceph-mon[67199]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-19T01:47:34.780 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:34 smithi092 ceph-mon[62574]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:47:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:34 smithi144 ceph-mon[67199]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:47:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:34 smithi159 ceph-mon[67110]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:47:36.755 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:36 smithi092 ceph-mon[62574]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-19T01:47:36.756 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:36 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-19T01:47:36.756 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:36 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:36 smithi144 ceph-mon[67199]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-19T01:47:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:36 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-19T01:47:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:36 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:36 smithi159 ceph-mon[67110]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-19T01:47:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:36 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-19T01:47:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:36 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:37.740 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:37 smithi092 ceph-mon[62574]: Deploying daemon osd.6 on smithi092 2024-03-19T01:47:37.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:37 smithi144 ceph-mon[67199]: Deploying daemon osd.6 on smithi092 2024-03-19T01:47:37.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:37 smithi159 ceph-mon[67110]: Deploying daemon osd.6 on smithi092 2024-03-19T01:47:38.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:38 smithi092 ceph-mon[62574]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:47:38.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:38 smithi144 ceph-mon[67199]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:47:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:38 smithi159 ceph-mon[67110]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:47:40.716 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:40 smithi092 ceph-mon[62574]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-19T01:47:40.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:40 smithi144 ceph-mon[67199]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-19T01:47:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:40 smithi159 ceph-mon[67110]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-19T01:47:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:42 smithi092 ceph-mon[62574]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:42 smithi159 ceph-mon[67110]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:42 smithi144 ceph-mon[67199]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:44.906 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:44 smithi092 ceph-mon[62574]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:44 smithi159 ceph-mon[67110]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:44 smithi144 ceph-mon[67199]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:45.693 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 6 on host 'smithi092' 2024-03-19T01:47:45.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.693+0000 7fe04d7fa700 1 -- 172.21.15.92:0/3617107839 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe050062750 con 0x7fe0380a2910 2024-03-19T01:47:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0380a2910 msgr2=0x7fe0380a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0380a2910 0x7fe0380a4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe044000c00 tx=0x7fe044003960 comp rx=0 tx=0).stop 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050100640 msgr2=0x7fe050198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050100640 0x7fe050198510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe04000b530 tx=0x7fe04000b840 comp rx=0 tx=0).stop 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 shutdown_connections 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0380a2910 0x7fe0380a4dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe050104060 0x7fe05019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe050100640 0x7fe050198510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 --2- 172.21.15.92:0/3617107839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501036c0 0x7fe050198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 >> 172.21.15.92:0/3617107839 conn(0x7fe0500fbfa0 msgr2=0x7fe050106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 shutdown_connections 2024-03-19T01:47:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:47:45.695+0000 7fe0567c2700 1 -- 172.21.15.92:0/3617107839 wait complete. 2024-03-19T01:47:46.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:46.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:46.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:46.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-19T01:47:46.015 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.016 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.016 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:46.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:46.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:46.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:46.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-19T01:47:46.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:46.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:46.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:46.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:46.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-19T01:47:46.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:46.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:47:46.424 DEBUG:teuthology.orchestra.run.smithi092:osd.6> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.6.service 2024-03-19T01:47:46.426 INFO:tasks.cephadm:Deploying osd.7 on smithi144 with /dev/vg_nvme/lv_2... 2024-03-19T01:47:46.426 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-19T01:47:46.671 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:47:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:46 smithi092 ceph-mon[62574]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-19T01:47:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:46 smithi092 ceph-mon[62574]: osdmap e44: 7 total, 6 up, 7 in 2024-03-19T01:47:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:46 smithi092 ceph-mon[62574]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:47:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:46 smithi144 ceph-mon[67199]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-19T01:47:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:46 smithi144 ceph-mon[67199]: osdmap e44: 7 total, 6 up, 7 in 2024-03-19T01:47:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:46 smithi144 ceph-mon[67199]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:47:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:47.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:46 smithi159 ceph-mon[67110]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-19T01:47:47.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:46 smithi159 ceph-mon[67110]: osdmap e44: 7 total, 6 up, 7 in 2024-03-19T01:47:47.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:46 smithi159 ceph-mon[67110]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:47:47.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:47.560 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 01:47:47 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T01:47:47.299+0000 7fd1a3c7c700 -1 osd.6 0 waiting for initial osdmap 2024-03-19T01:47:47.561 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 01:47:47 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T01:47:47.310+0000 7fd19ba67700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:47:47.893 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:47 smithi092 ceph-mon[62574]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:47:47.893 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:47 smithi092 ceph-mon[62574]: osdmap e45: 7 total, 6 up, 7 in 2024-03-19T01:47:47.893 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:47.893 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:47.893 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:47 smithi092 ceph-mon[62574]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' 2024-03-19T01:47:47.893 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:47 smithi092 ceph-mon[62574]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:48.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:47 smithi144 ceph-mon[67199]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:47:48.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:47 smithi144 ceph-mon[67199]: osdmap e45: 7 total, 6 up, 7 in 2024-03-19T01:47:48.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:48.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:48.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:47 smithi144 ceph-mon[67199]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' 2024-03-19T01:47:48.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:47 smithi144 ceph-mon[67199]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:48.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:47 smithi159 ceph-mon[67110]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:47:48.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:47 smithi159 ceph-mon[67110]: osdmap e45: 7 total, 6 up, 7 in 2024-03-19T01:47:48.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:48.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:48.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:47 smithi159 ceph-mon[67110]: from='osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802]' entity='osd.6' 2024-03-19T01:47:48.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:47 smithi159 ceph-mon[67110]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-19T01:47:48.843 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-19T01:47:48.843 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-19T01:47:48.844 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-19T01:47:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-19T01:47:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013701 s, 765 MB/s 2024-03-19T01:47:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-19T01:47:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:48 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:47:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:48 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:47:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:48 smithi144 ceph-mon[67199]: osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] boot 2024-03-19T01:47:49.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:48 smithi144 ceph-mon[67199]: osdmap e46: 7 total, 7 up, 7 in 2024-03-19T01:47:49.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:49.191 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:48 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:47:49.191 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:48 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:47:49.191 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:49.191 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:48 smithi092 ceph-mon[62574]: osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] boot 2024-03-19T01:47:49.191 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:48 smithi092 ceph-mon[62574]: osdmap e46: 7 total, 7 up, 7 in 2024-03-19T01:47:49.191 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:49.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:48 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:47:49.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:48 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:47:49.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:49.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:48 smithi159 ceph-mon[67110]: osd.6 [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] boot 2024-03-19T01:47:49.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:48 smithi159 ceph-mon[67110]: osdmap e46: 7 total, 7 up, 7 in 2024-03-19T01:47:49.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-19T01:47:49.504 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-03-19T01:47:49.747 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:47:50.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: osdmap e47: 7 total, 7 up, 7 in 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:50.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: osdmap e47: 7 total, 7 up, 7 in 2024-03-19T01:47:50.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:50.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:50.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:50.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:50.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:50.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:50.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: osdmap e47: 7 total, 7 up, 7 in 2024-03-19T01:47:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:47:50.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:50.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:50.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:47:50.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:47:51.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.480+0000 7f9a77433700 1 -- 172.21.15.144:0/2417574562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102dd0 msgr2=0x7f9a70109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:51.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.480+0000 7f9a77433700 1 --2- 172.21.15.144:0/2417574562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102dd0 0x7f9a70109680 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800b0d0 tx=0x7f9a6800b3e0 comp rx=0 tx=0).stop 2024-03-19T01:47:51.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.480+0000 7f9a77433700 1 -- 172.21.15.144:0/2417574562 shutdown_connections 2024-03-19T01:47:51.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.480+0000 7f9a77433700 1 --2- 172.21.15.144:0/2417574562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102dd0 0x7f9a70109680 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:51.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.480+0000 7f9a77433700 1 --2- 172.21.15.144:0/2417574562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70101230 0x7f9a70101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:51.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.480+0000 7f9a77433700 1 --2- 172.21.15.144:0/2417574562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a70102430 0x7f9a70102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:51.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.480+0000 7f9a77433700 1 -- 172.21.15.144:0/2417574562 >> 172.21.15.144:0/2417574562 conn(0x7f9a700fc7d0 msgr2=0x7f9a700fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:47:51.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.480+0000 7f9a77433700 1 -- 172.21.15.144:0/2417574562 shutdown_connections 2024-03-19T01:47:51.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.480+0000 7f9a77433700 1 -- 172.21.15.144:0/2417574562 wait complete. 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a77433700 1 Processor -- start 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a77433700 1 -- start start 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a77433700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a70101230 0x7f9a7006b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a77433700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102430 0x7f9a7019d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a77433700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70102dd0 0x7f9a7019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a77433700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010cdf0 con 0x7f9a70102dd0 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a77433700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010cc70 con 0x7f9a70102430 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a77433700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010cf70 con 0x7f9a70101230 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a749ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102430 0x7f9a7019d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a749ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102430 0x7f9a7019d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39782/0 (socket says 172.21.15.144:39782) 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a749ce700 1 -- 172.21.15.144:0/680525551 learned_addr learned my addr 172.21.15.144:0/680525551 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a751cf700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a70101230 0x7f9a7006b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a759d0700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70102dd0 0x7f9a7019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a749ce700 1 -- 172.21.15.144:0/680525551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a70101230 msgr2=0x7f9a7006b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:51.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a749ce700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a70101230 0x7f9a7006b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a749ce700 1 -- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70102dd0 msgr2=0x7f9a7019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a749ce700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70102dd0 0x7f9a7019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a749ce700 1 -- 172.21.15.144:0/680525551 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a701a0110 con 0x7f9a70102430 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a751cf700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a70101230 0x7f9a7006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a759d0700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70102dd0 0x7f9a7019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.481+0000 7f9a749ce700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102430 0x7f9a7019d200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000eb30 tx=0x7f9a6000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.482+0000 7f9a667fc700 1 -- 172.21.15.144:0/680525551 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6000cc90 con 0x7f9a70102430 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.482+0000 7f9a667fc700 1 -- 172.21.15.144:0/680525551 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a60004500 con 0x7f9a70102430 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.482+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a701a0400 con 0x7f9a70102430 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.482+0000 7f9a667fc700 1 -- 172.21.15.144:0/680525551 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60010430 con 0x7f9a70102430 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.482+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a701a0940 con 0x7f9a70102430 2024-03-19T01:47:51.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.483+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a70068fd0 con 0x7f9a70102430 2024-03-19T01:47:51.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.484+0000 7f9a667fc700 1 -- 172.21.15.144:0/680525551 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f9a60003f70 con 0x7f9a70102430 2024-03-19T01:47:51.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.484+0000 7f9a667fc700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9a5c0a2890 0x7f9a5c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:47:51.486 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:51 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi092 2024-03-19T01:47:51.486 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:51 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi092 to 5696M 2024-03-19T01:47:51.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.485+0000 7f9a667fc700 1 -- 172.21.15.144:0/680525551 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f9a600ce170 con 0x7f9a70102430 2024-03-19T01:47:51.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.485+0000 7f9a751cf700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9a5c0a2890 0x7f9a5c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:47:51.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.485+0000 7f9a751cf700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9a5c0a2890 0x7f9a5c0a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a70102290 tx=0x7f9a6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:47:51.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.488+0000 7f9a667fc700 1 -- 172.21.15.144:0/680525551 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9a6001f340 con 0x7f9a70102430 2024-03-19T01:47:51.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:47:51.691+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9a70062750 con 0x7f9a5c0a2890 2024-03-19T01:47:51.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:51 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi092 2024-03-19T01:47:51.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:51 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi092 to 5696M 2024-03-19T01:47:51.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:51 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi092 2024-03-19T01:47:51.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:51 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi092 to 5696M 2024-03-19T01:47:52.541 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:52 smithi144 ceph-mon[67199]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:52.542 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:52 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:47:52.542 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:52 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:47:52.542 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:52 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:52.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:52 smithi159 ceph-mon[67110]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:52.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:52 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:47:52.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:52 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:47:52.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:52 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:52 smithi092 ceph-mon[62574]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:52 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:47:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:52 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:47:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:52 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:47:53.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:53 smithi144 ceph-mon[67199]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:47:53.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:53 smithi159 ceph-mon[67110]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:47:53.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:53 smithi092 ceph-mon[62574]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:47:54.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:54 smithi144 ceph-mon[67199]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:54.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:54 smithi159 ceph-mon[67110]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:54.711 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:54 smithi092 ceph-mon[62574]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:55.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:55 smithi144 ceph-mon[67199]: from='client.? 172.21.15.144:0/2854177813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64c441ee-0a42-4022-bb95-ceef4053acac"}]: dispatch 2024-03-19T01:47:55.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:55 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64c441ee-0a42-4022-bb95-ceef4053acac"}]: dispatch 2024-03-19T01:47:55.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:55 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64c441ee-0a42-4022-bb95-ceef4053acac"}]': finished 2024-03-19T01:47:55.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:55 smithi144 ceph-mon[67199]: osdmap e48: 8 total, 7 up, 8 in 2024-03-19T01:47:55.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:55 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:47:55.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:55 smithi144 ceph-mon[67199]: from='client.? 172.21.15.144:0/2171285156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:47:55.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:55 smithi159 ceph-mon[67110]: from='client.? 172.21.15.144:0/2854177813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64c441ee-0a42-4022-bb95-ceef4053acac"}]: dispatch 2024-03-19T01:47:55.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:55 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64c441ee-0a42-4022-bb95-ceef4053acac"}]: dispatch 2024-03-19T01:47:55.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:55 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64c441ee-0a42-4022-bb95-ceef4053acac"}]': finished 2024-03-19T01:47:55.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:55 smithi159 ceph-mon[67110]: osdmap e48: 8 total, 7 up, 8 in 2024-03-19T01:47:55.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:55 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:47:55.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:55 smithi159 ceph-mon[67110]: from='client.? 172.21.15.144:0/2171285156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:47:55.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:55 smithi092 ceph-mon[62574]: from='client.? 172.21.15.144:0/2854177813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64c441ee-0a42-4022-bb95-ceef4053acac"}]: dispatch 2024-03-19T01:47:55.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:55 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64c441ee-0a42-4022-bb95-ceef4053acac"}]: dispatch 2024-03-19T01:47:55.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:55 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64c441ee-0a42-4022-bb95-ceef4053acac"}]': finished 2024-03-19T01:47:55.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:55 smithi092 ceph-mon[62574]: osdmap e48: 8 total, 7 up, 8 in 2024-03-19T01:47:55.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:55 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:47:55.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:55 smithi092 ceph-mon[62574]: from='client.? 172.21.15.144:0/2171285156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:47:56.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:56 smithi144 ceph-mon[67199]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:56.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:56 smithi159 ceph-mon[67110]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:56.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:56 smithi092 ceph-mon[62574]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:58.802 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:47:58 smithi144 ceph-mon[67199]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:58.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:47:58 smithi092 ceph-mon[62574]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:47:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:47:58 smithi159 ceph-mon[67110]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:00.788 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:00 smithi144 ceph-mon[67199]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:00 smithi092 ceph-mon[62574]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:00 smithi159 ceph-mon[67110]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:02.721 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:02 smithi144 ceph-mon[67199]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:02.721 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:02 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-19T01:48:02.721 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:02 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:02.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:02 smithi092 ceph-mon[62574]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:02.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:02 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-19T01:48:02.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:02 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:02 smithi159 ceph-mon[67110]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:02 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-19T01:48:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:02 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:03.549 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:03 smithi144 ceph-mon[67199]: Deploying daemon osd.7 on smithi144 2024-03-19T01:48:03.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:03 smithi092 ceph-mon[62574]: Deploying daemon osd.7 on smithi144 2024-03-19T01:48:03.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:03 smithi159 ceph-mon[67110]: Deploying daemon osd.7 on smithi144 2024-03-19T01:48:04.794 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:04 smithi144 ceph-mon[67199]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:04.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:04 smithi092 ceph-mon[62574]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:04 smithi159 ceph-mon[67110]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:06.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:06 smithi092 ceph-mon[62574]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:06.823 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:06 smithi144 ceph-mon[67199]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:06 smithi159 ceph-mon[67110]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:08.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:08 smithi144 ceph-mon[67199]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:08.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:08.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:08.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:08 smithi092 ceph-mon[62574]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:08 smithi159 ceph-mon[67110]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:10.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:10 smithi144 ceph-mon[67199]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:10.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:10 smithi092 ceph-mon[62574]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:10 smithi159 ceph-mon[67110]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:11.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.257+0000 7f9a667fc700 1 -- 172.21.15.144:0/680525551 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9a70062750 con 0x7f9a5c0a2890 2024-03-19T01:48:11.258 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 7 on host 'smithi144' 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9a5c0a2890 msgr2=0x7f9a5c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9a5c0a2890 0x7f9a5c0a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a70102290 tx=0x7f9a6c00b410 comp rx=0 tx=0).stop 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102430 msgr2=0x7f9a7019d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102430 0x7f9a7019d200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000eb30 tx=0x7f9a6000ee40 comp rx=0 tx=0).stop 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 shutdown_connections 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9a5c0a2890 0x7f9a5c0a4d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a70102dd0 0x7f9a7019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a70101230 0x7f9a7006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 --2- 172.21.15.144:0/680525551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70102430 0x7f9a7019d200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 >> 172.21.15.144:0/680525551 conn(0x7f9a700fc7d0 msgr2=0x7f9a701056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.259+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 shutdown_connections 2024-03-19T01:48:11.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:48:11.260+0000 7f9a77433700 1 -- 172.21.15.144:0/680525551 wait complete. 2024-03-19T01:48:11.983 DEBUG:teuthology.orchestra.run.smithi144:osd.7> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.7.service 2024-03-19T01:48:11.986 INFO:tasks.cephadm:Deploying osd.8 on smithi159 with /dev/vg_nvme/lv_2... 2024-03-19T01:48:11.986 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-19T01:48:12.236 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:48:12.358 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.358 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.358 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:12 smithi159 ceph-mon[67110]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:12.358 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.358 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.358 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:12 smithi159 ceph-mon[67110]: from='osd.7 [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:48:12.358 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:12 smithi159 ceph-mon[67110]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:48:12.497 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.497 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.497 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:12 smithi144 ceph-mon[67199]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:12.497 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.497 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.497 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:12 smithi144 ceph-mon[67199]: from='osd.7 [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:48:12.497 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:12 smithi144 ceph-mon[67199]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:48:12.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:12 smithi092 ceph-mon[62574]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:12.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:12 smithi092 ceph-mon[62574]: from='osd.7 [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:48:12.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:12 smithi092 ceph-mon[62574]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-19T01:48:13.660 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:13 smithi159 ceph-mon[67110]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-19T01:48:13.660 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:13 smithi159 ceph-mon[67110]: osdmap e49: 8 total, 7 up, 8 in 2024-03-19T01:48:13.660 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:13.660 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:13 smithi159 ceph-mon[67110]: from='osd.7 [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:48:13.660 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:13 smithi159 ceph-mon[67110]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:48:13.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:13 smithi092 ceph-mon[62574]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-19T01:48:13.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:13 smithi092 ceph-mon[62574]: osdmap e49: 8 total, 7 up, 8 in 2024-03-19T01:48:13.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:13.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:13 smithi092 ceph-mon[62574]: from='osd.7 [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:48:13.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:13 smithi092 ceph-mon[62574]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:48:13.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:13 smithi144 ceph-mon[67199]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-19T01:48:13.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:13 smithi144 ceph-mon[67199]: osdmap e49: 8 total, 7 up, 8 in 2024-03-19T01:48:13.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:13.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:13 smithi144 ceph-mon[67199]: from='osd.7 [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:48:13.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:13 smithi144 ceph-mon[67199]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:48:14.249 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 01:48:13 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T01:48:13.983+0000 7f96c265a700 -1 osd.7 0 waiting for initial osdmap 2024-03-19T01:48:14.249 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 01:48:13 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T01:48:13.993+0000 7f96ba445700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:48:14.375 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-19T01:48:14.375 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-19T01:48:14.375 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-19T01:48:14.376 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-19T01:48:14.376 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130549 s, 803 MB/s 2024-03-19T01:48:14.376 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-19T01:48:14.685 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:14 smithi159 ceph-mon[67110]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:14.685 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:14 smithi159 ceph-mon[67110]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:48:14.685 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:14 smithi159 ceph-mon[67110]: osdmap e50: 8 total, 7 up, 8 in 2024-03-19T01:48:14.685 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:14 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:14.685 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:14 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:14.685 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:14 smithi159 ceph-mon[67110]: from='osd.7 ' entity='osd.7' 2024-03-19T01:48:14.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:14 smithi092 ceph-mon[62574]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:14.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:14 smithi092 ceph-mon[62574]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:48:14.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:14 smithi092 ceph-mon[62574]: osdmap e50: 8 total, 7 up, 8 in 2024-03-19T01:48:14.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:14 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:14.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:14 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:14.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:14 smithi092 ceph-mon[62574]: from='osd.7 ' entity='osd.7' 2024-03-19T01:48:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:14 smithi144 ceph-mon[67199]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:14 smithi144 ceph-mon[67199]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:48:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:14 smithi144 ceph-mon[67199]: osdmap e50: 8 total, 7 up, 8 in 2024-03-19T01:48:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:14 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:14 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:14 smithi144 ceph-mon[67199]: from='osd.7 ' entity='osd.7' 2024-03-19T01:48:15.095 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2024-03-19T01:48:15.344 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:48:15.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:48:15.737 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: osd.7 [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] boot 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: osdmap e51: 8 total, 8 up, 8 in 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:15.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:15.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:15.739 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:48:15.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: osd.7 [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] boot 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: osdmap e51: 8 total, 8 up, 8 in 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:15.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:15.812 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:15.812 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:48:15.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: osd.7 [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] boot 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: osdmap e51: 8 total, 8 up, 8 in 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:15.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:15.848 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:16.804 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:16 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi144 2024-03-19T01:48:16.804 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:16 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi144 to 5693M 2024-03-19T01:48:16.804 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:16 smithi159 ceph-mon[67110]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:16.804 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:16 smithi159 ceph-mon[67110]: osdmap e52: 8 total, 8 up, 8 in 2024-03-19T01:48:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:16 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi144 2024-03-19T01:48:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:16 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi144 to 5693M 2024-03-19T01:48:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:16 smithi092 ceph-mon[62574]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:16 smithi092 ceph-mon[62574]: osdmap e52: 8 total, 8 up, 8 in 2024-03-19T01:48:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:16 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi144 2024-03-19T01:48:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:16 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi144 to 5693M 2024-03-19T01:48:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:16 smithi144 ceph-mon[67199]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-19T01:48:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:16 smithi144 ceph-mon[67199]: osdmap e52: 8 total, 8 up, 8 in 2024-03-19T01:48:17.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.038+0000 7f7300e9b700 1 -- 172.21.15.159:0/1944903833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc108030 msgr2=0x7f72fc10a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:17.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.038+0000 7f7300e9b700 1 --2- 172.21.15.159:0/1944903833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc108030 0x7f72fc10a4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f72f400b0a0 tx=0x7f72f400b3b0 comp rx=0 tx=0).stop 2024-03-19T01:48:17.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.038+0000 7f7300e9b700 1 -- 172.21.15.159:0/1944903833 shutdown_connections 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.038+0000 7f7300e9b700 1 --2- 172.21.15.159:0/1944903833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc108030 0x7f72fc10a4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.038+0000 7f7300e9b700 1 --2- 172.21.15.159:0/1944903833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72fc102dd0 0x7f72fc1051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.038+0000 7f7300e9b700 1 --2- 172.21.15.159:0/1944903833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc105700 0x7f72fc107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.038+0000 7f7300e9b700 1 -- 172.21.15.159:0/1944903833 >> 172.21.15.159:0/1944903833 conn(0x7f72fc0fc7e0 msgr2=0x7f72fc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.038+0000 7f7300e9b700 1 -- 172.21.15.159:0/1944903833 shutdown_connections 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.038+0000 7f7300e9b700 1 -- 172.21.15.159:0/1944903833 wait complete. 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f7300e9b700 1 Processor -- start 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f7300e9b700 1 -- start start 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f7300e9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc102dd0 0x7f72fc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f7300e9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc105700 0x7f72fc198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f7300e9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72fc108030 0x7f72fc19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f7300e9b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc10df70 con 0x7f72fc105700 2024-03-19T01:48:17.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f7300e9b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc10ddf0 con 0x7f72fc108030 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f7300e9b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc10e0f0 con 0x7f72fc102dd0 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72fa59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc102dd0 0x7f72fc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72fa59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc102dd0 0x7f72fc1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54514/0 (socket says 172.21.15.159:54514) 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72fa59c700 1 -- 172.21.15.159:0/3948552778 learned_addr learned my addr 172.21.15.159:0/3948552778 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72f9d9b700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc105700 0x7f72fc198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72fad9d700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72fc108030 0x7f72fc19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72fa59c700 1 -- 172.21.15.159:0/3948552778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72fc108030 msgr2=0x7f72fc19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72fa59c700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72fc108030 0x7f72fc19fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72fa59c700 1 -- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc105700 msgr2=0x7f72fc198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72fa59c700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc105700 0x7f72fc198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.039+0000 7f72fa59c700 1 -- 172.21.15.159:0/3948552778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc100bc0 con 0x7f72fc102dd0 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.040+0000 7f72fa59c700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc102dd0 0x7f72fc1985b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f72f000b560 tx=0x7f72f000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:48:17.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.040+0000 7f72eb7fe700 1 -- 172.21.15.159:0/3948552778 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0015020 con 0x7f72fc102dd0 2024-03-19T01:48:17.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.040+0000 7f72eb7fe700 1 -- 172.21.15.159:0/3948552778 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f0011d10 con 0x7f72fc102dd0 2024-03-19T01:48:17.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.040+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc100eb0 con 0x7f72fc102dd0 2024-03-19T01:48:17.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.040+0000 7f72eb7fe700 1 -- 172.21.15.159:0/3948552778 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0010e50 con 0x7f72fc102dd0 2024-03-19T01:48:17.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.040+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc1013a0 con 0x7f72fc102dd0 2024-03-19T01:48:17.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.041+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72fc068fd0 con 0x7f72fc102dd0 2024-03-19T01:48:17.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.042+0000 7f72eb7fe700 1 -- 172.21.15.159:0/3948552778 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f72f0018020 con 0x7f72fc102dd0 2024-03-19T01:48:17.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.043+0000 7f72eb7fe700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72e40a2a30 0x7f72e40a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:48:17.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.043+0000 7f72eb7fe700 1 -- 172.21.15.159:0/3948552778 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f72f00cdb70 con 0x7f72fc102dd0 2024-03-19T01:48:17.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.043+0000 7f72f9d9b700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72e40a2a30 0x7f72e40a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:48:17.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.043+0000 7f72f9d9b700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72e40a2a30 0x7f72e40a4ef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72ec005950 tx=0x7f72ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:48:17.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.047+0000 7f72eb7fe700 1 -- 172.21.15.159:0/3948552778 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f72f001f020 con 0x7f72fc102dd0 2024-03-19T01:48:17.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:17.251+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f72fc0fdef0 con 0x7f72e40a2a30 2024-03-19T01:48:17.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:17 smithi144 ceph-mon[67199]: osdmap e53: 8 total, 8 up, 8 in 2024-03-19T01:48:17.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:17 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:48:17.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:17 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:48:17.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:17 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:17.855 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:17 smithi159 ceph-mon[67110]: osdmap e53: 8 total, 8 up, 8 in 2024-03-19T01:48:17.855 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:17 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:48:17.855 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:17 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:48:17.855 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:17 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:18.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:17 smithi092 ceph-mon[62574]: osdmap e53: 8 total, 8 up, 8 in 2024-03-19T01:48:18.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:17 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:48:18.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:17 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:48:18.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:17 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:18.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:18 smithi144 ceph-mon[67199]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:48:18.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:18 smithi144 ceph-mon[67199]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:18.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:18 smithi144 ceph-mon[67199]: osdmap e54: 8 total, 8 up, 8 in 2024-03-19T01:48:18.919 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:18 smithi159 ceph-mon[67110]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:48:18.919 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:18 smithi159 ceph-mon[67110]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:18.919 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:18 smithi159 ceph-mon[67110]: osdmap e54: 8 total, 8 up, 8 in 2024-03-19T01:48:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:18 smithi092 ceph-mon[62574]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:48:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:18 smithi092 ceph-mon[62574]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:18 smithi092 ceph-mon[62574]: osdmap e54: 8 total, 8 up, 8 in 2024-03-19T01:48:19.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:19 smithi144 ceph-mon[67199]: osdmap e55: 8 total, 8 up, 8 in 2024-03-19T01:48:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:19 smithi159 ceph-mon[67110]: osdmap e55: 8 total, 8 up, 8 in 2024-03-19T01:48:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:19 smithi092 ceph-mon[62574]: osdmap e55: 8 total, 8 up, 8 in 2024-03-19T01:48:20.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:20 smithi144 ceph-mon[67199]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:20.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:20 smithi144 ceph-mon[67199]: from='client.? 172.21.15.159:0/2992577629' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91"}]: dispatch 2024-03-19T01:48:20.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:20 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91"}]: dispatch 2024-03-19T01:48:20.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:20 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91"}]': finished 2024-03-19T01:48:20.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:20 smithi144 ceph-mon[67199]: osdmap e56: 9 total, 8 up, 9 in 2024-03-19T01:48:20.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:20.856 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:20 smithi159 ceph-mon[67110]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:20.856 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:20 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/2992577629' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91"}]: dispatch 2024-03-19T01:48:20.856 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:20 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91"}]: dispatch 2024-03-19T01:48:20.856 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:20 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91"}]': finished 2024-03-19T01:48:20.856 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:20 smithi159 ceph-mon[67110]: osdmap e56: 9 total, 8 up, 9 in 2024-03-19T01:48:20.856 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:20 smithi092 ceph-mon[62574]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:20 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/2992577629' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91"}]: dispatch 2024-03-19T01:48:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:20 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91"}]: dispatch 2024-03-19T01:48:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:20 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91"}]': finished 2024-03-19T01:48:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:20 smithi092 ceph-mon[62574]: osdmap e56: 9 total, 8 up, 9 in 2024-03-19T01:48:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:21.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:21 smithi144 ceph-mon[67199]: from='client.? 172.21.15.159:0/1058775117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:48:21.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:21 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/1058775117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:48:22.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:21 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/1058775117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:48:22.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:22 smithi144 ceph-mon[67199]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:22 smithi159 ceph-mon[67110]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:22 smithi092 ceph-mon[62574]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:24.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:24 smithi144 ceph-mon[67199]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:24.862 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:24 smithi092 ceph-mon[62574]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:24 smithi159 ceph-mon[67110]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:26.880 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:26 smithi159 ceph-mon[67110]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:26 smithi092 ceph-mon[62574]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:26 smithi144 ceph-mon[67199]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:28 smithi159 ceph-mon[67110]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:28 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-19T01:48:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:28 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:28 smithi159 ceph-mon[67110]: Deploying daemon osd.8 on smithi159 2024-03-19T01:48:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:28 smithi092 ceph-mon[62574]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:28 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-19T01:48:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:28 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:28 smithi092 ceph-mon[62574]: Deploying daemon osd.8 on smithi159 2024-03-19T01:48:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:28 smithi144 ceph-mon[67199]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:28 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-19T01:48:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:28 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:28 smithi144 ceph-mon[67199]: Deploying daemon osd.8 on smithi159 2024-03-19T01:48:30.676 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:30 smithi159 ceph-mon[67110]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:30 smithi092 ceph-mon[62574]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:30 smithi144 ceph-mon[67199]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:32.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:32 smithi159 ceph-mon[67110]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:32 smithi092 ceph-mon[62574]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:32 smithi144 ceph-mon[67199]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:34 smithi092 ceph-mon[62574]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:34 smithi144 ceph-mon[67199]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:34.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:34 smithi159 ceph-mon[67110]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:34.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:34.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:34.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:36 smithi092 ceph-mon[62574]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:36 smithi144 ceph-mon[67199]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:36.892 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:36 smithi159 ceph-mon[67110]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:36.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.965+0000 7f72eb7fe700 1 -- 172.21.15.159:0/3948552778 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f72fc0fdef0 con 0x7f72e40a2a30 2024-03-19T01:48:36.966 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 8 on host 'smithi159' 2024-03-19T01:48:36.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72e40a2a30 msgr2=0x7f72e40a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72e40a2a30 0x7f72e40a4ef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72ec005950 tx=0x7f72ec00b410 comp rx=0 tx=0).stop 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc102dd0 msgr2=0x7f72fc1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc102dd0 0x7f72fc1985b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f72f000b560 tx=0x7f72f000b870 comp rx=0 tx=0).stop 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 shutdown_connections 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72e40a2a30 0x7f72e40a4ef0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72fc108030 0x7f72fc19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72fc102dd0 0x7f72fc1985b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 --2- 172.21.15.159:0/3948552778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc105700 0x7f72fc198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 >> 172.21.15.159:0/3948552778 conn(0x7f72fc0fc7e0 msgr2=0x7f72fc103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 shutdown_connections 2024-03-19T01:48:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:48:36.968+0000 7f7300e9b700 1 -- 172.21.15.159:0/3948552778 wait complete. 2024-03-19T01:48:38.000 DEBUG:teuthology.orchestra.run.smithi159:osd.8> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.8.service 2024-03-19T01:48:38.003 INFO:tasks.cephadm:Deploying osd.9 on smithi092 with /dev/vg_nvme/lv_1... 2024-03-19T01:48:38.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-19T01:48:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:37 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:37 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:37 smithi159 ceph-mon[67110]: from='osd.8 [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:48:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:37 smithi159 ceph-mon[67110]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:48:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:37 smithi159 ceph-mon[67110]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:38.249 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:48:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:37 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:37 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:37 smithi092 ceph-mon[62574]: from='osd.8 [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:48:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:37 smithi092 ceph-mon[62574]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:48:38.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:37 smithi092 ceph-mon[62574]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:38.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:37 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:38.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:37 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:38.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:37 smithi144 ceph-mon[67199]: from='osd.8 [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:48:38.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:37 smithi144 ceph-mon[67199]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-19T01:48:38.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:37 smithi144 ceph-mon[67199]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:39.070 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:38 smithi159 ceph-mon[67110]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-19T01:48:39.070 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:38 smithi159 ceph-mon[67110]: from='osd.8 [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:48:39.070 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:38 smithi159 ceph-mon[67110]: osdmap e57: 9 total, 8 up, 9 in 2024-03-19T01:48:39.070 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:38 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:39.070 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:38 smithi159 ceph-mon[67110]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:48:39.070 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:38 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:39.071 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:38 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:39.071 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 01:48:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T01:48:38.950+0000 7f40546b3700 -1 osd.8 0 waiting for initial osdmap 2024-03-19T01:48:39.071 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 01:48:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T01:48:38.959+0000 7f40504a6700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:48:39.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:38 smithi092 ceph-mon[62574]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-19T01:48:39.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:38 smithi092 ceph-mon[62574]: from='osd.8 [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:48:39.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:38 smithi092 ceph-mon[62574]: osdmap e57: 9 total, 8 up, 9 in 2024-03-19T01:48:39.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:38 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:39.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:38 smithi092 ceph-mon[62574]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:48:39.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:38 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:39.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:38 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:39.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:38 smithi144 ceph-mon[67199]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-19T01:48:39.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:38 smithi144 ceph-mon[67199]: from='osd.8 [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:48:39.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:38 smithi144 ceph-mon[67199]: osdmap e57: 9 total, 8 up, 9 in 2024-03-19T01:48:39.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:38 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:39.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:38 smithi144 ceph-mon[67199]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:48:39.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:38 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:39.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:38 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:40.016 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:39 smithi092 ceph-mon[62574]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:48:40.017 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:39 smithi092 ceph-mon[62574]: osdmap e58: 9 total, 8 up, 9 in 2024-03-19T01:48:40.017 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:40.017 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:40.017 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:39 smithi092 ceph-mon[62574]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:40.069 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:39 smithi159 ceph-mon[67110]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:48:40.069 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:39 smithi159 ceph-mon[67110]: osdmap e58: 9 total, 8 up, 9 in 2024-03-19T01:48:40.069 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:40.070 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:40.070 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:39 smithi159 ceph-mon[67110]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:39 smithi144 ceph-mon[67199]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:39 smithi144 ceph-mon[67199]: osdmap e58: 9 total, 8 up, 9 in 2024-03-19T01:48:40.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:40.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:40.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:39 smithi144 ceph-mon[67199]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-19T01:48:40.467 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-19T01:48:40.468 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-19T01:48:40.468 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-03-19T01:48:40.468 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-03-19T01:48:40.468 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011685 s, 897 MB/s 2024-03-19T01:48:40.468 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-03-19T01:48:41.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:40 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:48:41.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:40 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:48:41.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:41.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:40 smithi159 ceph-mon[67110]: osd.8 [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] boot 2024-03-19T01:48:41.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:40 smithi159 ceph-mon[67110]: osdmap e59: 9 total, 9 up, 9 in 2024-03-19T01:48:41.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:41.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:40 smithi159 ceph-mon[67110]: osdmap e60: 9 total, 9 up, 9 in 2024-03-19T01:48:41.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:40 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:48:41.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:40 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:48:41.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:41.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:40 smithi092 ceph-mon[62574]: osd.8 [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] boot 2024-03-19T01:48:41.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:40 smithi092 ceph-mon[62574]: osdmap e59: 9 total, 9 up, 9 in 2024-03-19T01:48:41.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:41.214 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:40 smithi092 ceph-mon[62574]: osdmap e60: 9 total, 9 up, 9 in 2024-03-19T01:48:41.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-03-19T01:48:41.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:40 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:48:41.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:40 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:48:41.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:41.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:40 smithi144 ceph-mon[67199]: osd.8 [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] boot 2024-03-19T01:48:41.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:40 smithi144 ceph-mon[67199]: osdmap e59: 9 total, 9 up, 9 in 2024-03-19T01:48:41.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-19T01:48:41.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:40 smithi144 ceph-mon[67199]: osdmap e60: 9 total, 9 up, 9 in 2024-03-19T01:48:41.510 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:48:42.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:42.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:41 smithi092 ceph-mon[62574]: osdmap e61: 9 total, 9 up, 9 in 2024-03-19T01:48:42.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:42.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:41 smithi144 ceph-mon[67199]: osdmap e61: 9 total, 9 up, 9 in 2024-03-19T01:48:42.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:42.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:48:42.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:41 smithi159 ceph-mon[67110]: osdmap e61: 9 total, 9 up, 9 in 2024-03-19T01:48:43.243 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:42 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi159 2024-03-19T01:48:43.243 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:42 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi159 to 5693M 2024-03-19T01:48:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.245+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/3228507736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8105700 msgr2=0x7ff5e8107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.245+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/3228507736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8105700 0x7ff5e8107af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009990 tx=0x7ff5d8009ca0 comp rx=0 tx=0).stop 2024-03-19T01:48:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.245+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/3228507736 shutdown_connections 2024-03-19T01:48:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.245+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/3228507736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8108030 0x7ff5e810a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.245+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/3228507736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5e8102dd0 0x7ff5e81051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.245+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/3228507736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8105700 0x7ff5e8107af0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.245+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/3228507736 >> 172.21.15.92:0/3228507736 conn(0x7ff5e80fc7e0 msgr2=0x7ff5e80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:48:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.245+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/3228507736 shutdown_connections 2024-03-19T01:48:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.245+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/3228507736 wait complete. 2024-03-19T01:48:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ef7fe700 1 Processor -- start 2024-03-19T01:48:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ef7fe700 1 -- start start 2024-03-19T01:48:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8102dd0 0x7ff5e8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:48:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5e8105700 0x7ff5e8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:48:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ef7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8108030 0x7ff5e819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:48:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ef7fe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e810df90 con 0x7ff5e8102dd0 2024-03-19T01:48:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ef7fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e810de10 con 0x7ff5e8108030 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ef7fe700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e810e110 con 0x7ff5e8105700 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ed59a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8102dd0 0x7ff5e8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ed59a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8102dd0 0x7ff5e8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49592/0 (socket says 172.21.15.92:49592) 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ed59a700 1 -- 172.21.15.92:0/2104001670 learned_addr learned my addr 172.21.15.92:0/2104001670 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5edd9b700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8108030 0x7ff5e819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ecd99700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5e8105700 0x7ff5e8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ed59a700 1 -- 172.21.15.92:0/2104001670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5e8105700 msgr2=0x7ff5e8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ed59a700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5e8105700 0x7ff5e8198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ed59a700 1 -- 172.21.15.92:0/2104001670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8108030 msgr2=0x7ff5e819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ed59a700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8108030 0x7ff5e819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:48:43.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5ed59a700 1 -- 172.21.15.92:0/2104001670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e81a0260 con 0x7ff5e8102dd0 2024-03-19T01:48:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.246+0000 7ff5edd9b700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8108030 0x7ff5e819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:48:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.247+0000 7ff5ed59a700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8102dd0 0x7ff5e8198600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400b560 tx=0x7ff5e400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:48:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.247+0000 7ff5de7fc700 1 -- 172.21.15.92:0/2104001670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4015020 con 0x7ff5e8102dd0 2024-03-19T01:48:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.247+0000 7ff5de7fc700 1 -- 172.21.15.92:0/2104001670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e4011d10 con 0x7ff5e8102dd0 2024-03-19T01:48:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.247+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e81a0550 con 0x7ff5e8102dd0 2024-03-19T01:48:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.247+0000 7ff5de7fc700 1 -- 172.21.15.92:0/2104001670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400f2e0 con 0x7ff5e8102dd0 2024-03-19T01:48:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.247+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e81a0b10 con 0x7ff5e8102dd0 2024-03-19T01:48:43.250 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.248+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e8068fd0 con 0x7ff5e8102dd0 2024-03-19T01:48:43.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.249+0000 7ff5de7fc700 1 -- 172.21.15.92:0/2104001670 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ff5e4018020 con 0x7ff5e8102dd0 2024-03-19T01:48:43.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.249+0000 7ff5de7fc700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5d40a2910 0x7ff5d40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:48:43.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.249+0000 7ff5ecd99700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5d40a2910 0x7ff5d40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:48:43.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.250+0000 7ff5de7fc700 1 -- 172.21.15.92:0/2104001670 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7ff5e40cdcd0 con 0x7ff5e8102dd0 2024-03-19T01:48:43.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.250+0000 7ff5ecd99700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5d40a2910 0x7ff5d40a4dd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80039c0 tx=0x7ff5d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:48:43.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.253+0000 7ff5de7fc700 1 -- 172.21.15.92:0/2104001670 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff5e401f020 con 0x7ff5e8102dd0 2024-03-19T01:48:43.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:42 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi159 2024-03-19T01:48:43.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:42 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi159 to 5693M 2024-03-19T01:48:43.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:42 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi159 2024-03-19T01:48:43.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:42 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi159 to 5693M 2024-03-19T01:48:43.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:48:43.459+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff5e8062750 con 0x7ff5d40a2910 2024-03-19T01:48:44.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:43 smithi092 ceph-mon[62574]: osdmap e62: 9 total, 9 up, 9 in 2024-03-19T01:48:44.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:43 smithi092 ceph-mon[62574]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:44.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:43 smithi092 ceph-mon[62574]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:48:44.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:48:44.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:48:44.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:43 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:44.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:43 smithi144 ceph-mon[67199]: osdmap e62: 9 total, 9 up, 9 in 2024-03-19T01:48:44.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:43 smithi144 ceph-mon[67199]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:44.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:43 smithi144 ceph-mon[67199]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:48:44.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:48:44.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:48:44.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:43 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:44.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:43 smithi159 ceph-mon[67110]: osdmap e62: 9 total, 9 up, 9 in 2024-03-19T01:48:44.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:43 smithi159 ceph-mon[67110]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:44.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:43 smithi159 ceph-mon[67110]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:48:44.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:48:44.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:48:44.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:43 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:44 smithi092 ceph-mon[62574]: osdmap e63: 9 total, 9 up, 9 in 2024-03-19T01:48:45.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:44 smithi144 ceph-mon[67199]: osdmap e63: 9 total, 9 up, 9 in 2024-03-19T01:48:45.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:44 smithi159 ceph-mon[67110]: osdmap e63: 9 total, 9 up, 9 in 2024-03-19T01:48:46.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:45 smithi092 ceph-mon[62574]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:46.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:45 smithi144 ceph-mon[67199]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:46.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:45 smithi159 ceph-mon[67110]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:47.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:46 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/194318424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cfda0d-94d8-4310-be83-3c0348ef3486"}]: dispatch 2024-03-19T01:48:47.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:46 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/194318424' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32cfda0d-94d8-4310-be83-3c0348ef3486"}]': finished 2024-03-19T01:48:47.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:46 smithi092 ceph-mon[62574]: osdmap e64: 10 total, 9 up, 10 in 2024-03-19T01:48:47.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:48:47.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:46 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/194318424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cfda0d-94d8-4310-be83-3c0348ef3486"}]: dispatch 2024-03-19T01:48:47.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:46 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/194318424' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32cfda0d-94d8-4310-be83-3c0348ef3486"}]': finished 2024-03-19T01:48:47.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:46 smithi144 ceph-mon[67199]: osdmap e64: 10 total, 9 up, 10 in 2024-03-19T01:48:47.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:48:47.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:46 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/194318424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32cfda0d-94d8-4310-be83-3c0348ef3486"}]: dispatch 2024-03-19T01:48:47.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:46 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/194318424' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32cfda0d-94d8-4310-be83-3c0348ef3486"}]': finished 2024-03-19T01:48:47.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:46 smithi159 ceph-mon[67110]: osdmap e64: 10 total, 9 up, 10 in 2024-03-19T01:48:47.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:48:48.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:47 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/318376425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:48:48.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:47 smithi092 ceph-mon[62574]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-03-19T01:48:48.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:47 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/318376425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:48:48.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:47 smithi144 ceph-mon[67199]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-03-19T01:48:48.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:47 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/318376425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:48:48.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:47 smithi159 ceph-mon[67110]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-03-19T01:48:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:50 smithi092 ceph-mon[62574]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-19T01:48:50.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:50 smithi144 ceph-mon[67199]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-19T01:48:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:50 smithi159 ceph-mon[67110]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-03-19T01:48:52.759 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:52 smithi092 ceph-mon[62574]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:48:52.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:52 smithi144 ceph-mon[67199]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:48:52.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:52 smithi159 ceph-mon[67110]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:48:54.699 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:54 smithi092 ceph-mon[62574]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-19T01:48:54.700 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-19T01:48:54.700 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:54 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:54.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:54 smithi144 ceph-mon[67199]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-19T01:48:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-19T01:48:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:54 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:54 smithi159 ceph-mon[67110]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-19T01:48:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-19T01:48:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:54 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:48:55.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:55 smithi092 ceph-mon[62574]: Deploying daemon osd.9 on smithi092 2024-03-19T01:48:55.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:55 smithi144 ceph-mon[67199]: Deploying daemon osd.9 on smithi092 2024-03-19T01:48:55.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:55 smithi159 ceph-mon[67110]: Deploying daemon osd.9 on smithi092 2024-03-19T01:48:56.576 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:56 smithi092 ceph-mon[62574]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:48:56.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:56 smithi144 ceph-mon[67199]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:48:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:56 smithi159 ceph-mon[67110]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-19T01:48:58.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:48:58 smithi092 ceph-mon[62574]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:58.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:48:58 smithi144 ceph-mon[67199]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:48:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:48:58 smithi159 ceph-mon[67110]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:00.617 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:00 smithi092 ceph-mon[62574]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:00.617 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:00.617 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:00.618 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:00 smithi144 ceph-mon[67199]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:00 smithi159 ceph-mon[67110]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:02.786 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:02 smithi092 ceph-mon[62574]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:02.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:02 smithi144 ceph-mon[67199]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:02 smithi159 ceph-mon[67110]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:03.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:03.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:03.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:03.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:03.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:03 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:03.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:03 smithi092 ceph-mon[62574]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:04.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.024+0000 7ff5de7fc700 1 -- 172.21.15.92:0/2104001670 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff5e8062750 con 0x7ff5d40a2910 2024-03-19T01:49:04.025 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 9 on host 'smithi092' 2024-03-19T01:49:04.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5d40a2910 msgr2=0x7ff5d40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5d40a2910 0x7ff5d40a4dd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80039c0 tx=0x7ff5d8005fb0 comp rx=0 tx=0).stop 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8102dd0 msgr2=0x7ff5e8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8102dd0 0x7ff5e8198600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400b560 tx=0x7ff5e400b920 comp rx=0 tx=0).stop 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 shutdown_connections 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5d40a2910 0x7ff5d40a4dd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8108030 0x7ff5e819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5e8102dd0 0x7ff5e8198600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/2104001670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5e8105700 0x7ff5e8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.026+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 >> 172.21.15.92:0/2104001670 conn(0x7ff5e80fc7e0 msgr2=0x7ff5e8103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.027+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 shutdown_connections 2024-03-19T01:49:04.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:04.027+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/2104001670 wait complete. 2024-03-19T01:49:04.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:03 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:04.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:03 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:04.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:03 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:04.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:03 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:04.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:03 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:04.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:03 smithi144 ceph-mon[67199]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:04.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:03 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:04.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:03 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:04.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:03 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:04.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:03 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:04.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:03 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:04.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:03 smithi159 ceph-mon[67110]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:04.807 DEBUG:teuthology.orchestra.run.smithi092:osd.9> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.9.service 2024-03-19T01:49:04.810 INFO:tasks.cephadm:Deploying osd.10 on smithi144 with /dev/vg_nvme/lv_1... 2024-03-19T01:49:04.810 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-19T01:49:05.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:04 smithi092 ceph-mon[62574]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-19T01:49:05.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:04 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:05.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:04 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:05.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:04 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:49:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:04 smithi144 ceph-mon[67199]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-19T01:49:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:04 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:04 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:05.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:04 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:05.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:04 smithi159 ceph-mon[67110]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-19T01:49:05.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:04 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:05.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:04 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:05.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:04 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:06.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:05 smithi092 ceph-mon[62574]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-19T01:49:06.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:05 smithi092 ceph-mon[62574]: osdmap e65: 10 total, 9 up, 10 in 2024-03-19T01:49:06.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:05 smithi092 ceph-mon[62574]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:49:06.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:05 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:06.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:05 smithi092 ceph-mon[62574]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:06.061 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 01:49:05 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T01:49:05.746+0000 7f235cde5700 -1 osd.9 0 waiting for initial osdmap 2024-03-19T01:49:06.062 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 01:49:05 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T01:49:05.758+0000 7f2358bd8700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:49:06.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:05 smithi144 ceph-mon[67199]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-19T01:49:06.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:05 smithi144 ceph-mon[67199]: osdmap e65: 10 total, 9 up, 10 in 2024-03-19T01:49:06.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:05 smithi144 ceph-mon[67199]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:49:06.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:05 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:06.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:05 smithi144 ceph-mon[67199]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:06.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:05 smithi159 ceph-mon[67110]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-19T01:49:06.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:05 smithi159 ceph-mon[67110]: osdmap e65: 10 total, 9 up, 10 in 2024-03-19T01:49:06.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:05 smithi159 ceph-mon[67110]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-03-19T01:49:06.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:05 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:06.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:05 smithi159 ceph-mon[67110]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-19T01:49:06.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:06 smithi092 ceph-mon[62574]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:49:06.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:06 smithi092 ceph-mon[62574]: osdmap e66: 10 total, 9 up, 10 in 2024-03-19T01:49:06.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:06 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:06.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:06 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:06 smithi144 ceph-mon[67199]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:49:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:06 smithi144 ceph-mon[67199]: osdmap e66: 10 total, 9 up, 10 in 2024-03-19T01:49:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:06 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:06 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:07.129 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-19T01:49:07.129 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-19T01:49:07.129 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-19T01:49:07.130 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-19T01:49:07.130 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124863 s, 840 MB/s 2024-03-19T01:49:07.130 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-19T01:49:07.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:06 smithi159 ceph-mon[67110]: from='osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-03-19T01:49:07.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:06 smithi159 ceph-mon[67110]: osdmap e66: 10 total, 9 up, 10 in 2024-03-19T01:49:07.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:06 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:07.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:06 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:07.908 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-03-19T01:49:08.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:07 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:49:08.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:07 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:49:08.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:07 smithi092 ceph-mon[62574]: osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] boot 2024-03-19T01:49:08.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:07 smithi092 ceph-mon[62574]: osdmap e67: 10 total, 10 up, 10 in 2024-03-19T01:49:08.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:07 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:08.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:07 smithi092 ceph-mon[62574]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:08.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:07 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:49:08.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:07 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:49:08.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:07 smithi144 ceph-mon[67199]: osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] boot 2024-03-19T01:49:08.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:07 smithi144 ceph-mon[67199]: osdmap e67: 10 total, 10 up, 10 in 2024-03-19T01:49:08.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:07 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:08.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:07 smithi144 ceph-mon[67199]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:08.148 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T01:49:08.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:07 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:49:08.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:07 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:49:08.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:07 smithi159 ceph-mon[67110]: osd.9 [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] boot 2024-03-19T01:49:08.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:07 smithi159 ceph-mon[67110]: osdmap e67: 10 total, 10 up, 10 in 2024-03-19T01:49:08.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:07 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-19T01:49:08.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:07 smithi159 ceph-mon[67110]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: osdmap e68: 10 total, 10 up, 10 in 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi092 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi092 to 4272M 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:08.974 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:08.975 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:08.975 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: osdmap e68: 10 total, 10 up, 10 in 2024-03-19T01:49:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi092 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi092 to 4272M 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:09.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:09.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: osdmap e68: 10 total, 10 up, 10 in 2024-03-19T01:49:09.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi092 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi092 to 4272M 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:09.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:09.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.840+0000 7f7844c8f700 1 -- 172.21.15.144:0/2776656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103fd0 msgr2=0x7f784010a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:09.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.840+0000 7f7844c8f700 1 --2- 172.21.15.144:0/2776656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103fd0 0x7f784010a880 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f783800b0a0 tx=0x7f783800b3b0 comp rx=0 tx=0).stop 2024-03-19T01:49:09.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.840+0000 7f7844c8f700 1 -- 172.21.15.144:0/2776656 shutdown_connections 2024-03-19T01:49:09.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.840+0000 7f7844c8f700 1 --2- 172.21.15.144:0/2776656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103fd0 0x7f784010a880 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:09.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.840+0000 7f7844c8f700 1 --2- 172.21.15.144:0/2776656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7840100650 0x7f7840100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:09.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.840+0000 7f7844c8f700 1 --2- 172.21.15.144:0/2776656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7840103630 0x7f7840103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:09.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.840+0000 7f7844c8f700 1 -- 172.21.15.144:0/2776656 >> 172.21.15.144:0/2776656 conn(0x7f78400fbfb0 msgr2=0x7f78400fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:49:09.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.840+0000 7f7844c8f700 1 -- 172.21.15.144:0/2776656 shutdown_connections 2024-03-19T01:49:09.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.840+0000 7f7844c8f700 1 -- 172.21.15.144:0/2776656 wait complete. 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f7844c8f700 1 Processor -- start 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f7844c8f700 1 -- start start 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f7844c8f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7840100650 0x7f78401985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f7844c8f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103630 0x7f7840198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f7844c8f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7840103fd0 0x7f784019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f7844c8f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f784010e060 con 0x7f7840100650 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f7844c8f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f784010dee0 con 0x7f7840103630 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f7844c8f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f784010e1e0 con 0x7f7840103fd0 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f783dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103630 0x7f7840198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:09.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f783dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103630 0x7f7840198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42914/0 (socket says 172.21.15.144:42914) 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f783dd9b700 1 -- 172.21.15.144:0/1160272815 learned_addr learned my addr 172.21.15.144:0/1160272815 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.841+0000 7f783ed9d700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7840103fd0 0x7f784019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f783e59c700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7840100650 0x7f78401985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f783dd9b700 1 -- 172.21.15.144:0/1160272815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7840103fd0 msgr2=0x7f784019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f783dd9b700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7840103fd0 0x7f784019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f783dd9b700 1 -- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7840100650 msgr2=0x7f78401985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f783dd9b700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7840100650 0x7f78401985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f783dd9b700 1 -- 172.21.15.144:0/1160272815 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78401a0210 con 0x7f7840103630 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f783e59c700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7840100650 0x7f78401985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f783dd9b700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103630 0x7f7840198af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f783000b530 tx=0x7f783000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f782f7fe700 1 -- 172.21.15.144:0/1160272815 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830015020 con 0x7f7840103630 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78401a0500 con 0x7f7840103630 2024-03-19T01:49:09.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f782f7fe700 1 -- 172.21.15.144:0/1160272815 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7830011d10 con 0x7f7840103630 2024-03-19T01:49:09.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f782f7fe700 1 -- 172.21.15.144:0/1160272815 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783000f2e0 con 0x7f7840103630 2024-03-19T01:49:09.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.842+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78401a0ac0 con 0x7f7840103630 2024-03-19T01:49:09.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.843+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7840068fd0 con 0x7f7840103630 2024-03-19T01:49:09.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.844+0000 7f782f7fe700 1 -- 172.21.15.144:0/1160272815 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7830018020 con 0x7f7840103630 2024-03-19T01:49:09.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.845+0000 7f782f7fe700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78280a2890 0x7f78280a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:09.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.845+0000 7f782f7fe700 1 -- 172.21.15.144:0/1160272815 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f78300cdfc0 con 0x7f7840103630 2024-03-19T01:49:09.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:09 smithi144 ceph-mon[67199]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:09.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.845+0000 7f783e59c700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78280a2890 0x7f78280a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:09.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.845+0000 7f783e59c700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78280a2890 0x7f78280a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f784006a810 tx=0x7f7834008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:49:09.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:09.849+0000 7f782f7fe700 1 -- 172.21.15.144:0/1160272815 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f783001f020 con 0x7f7840103630 2024-03-19T01:49:10.053 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:10.053+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7840062750 con 0x7f78280a2890 2024-03-19T01:49:10.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:09 smithi092 ceph-mon[62574]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:10.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:09 smithi159 ceph-mon[67110]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:10.966 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:10 smithi144 ceph-mon[67199]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:49:10.966 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:49:10.966 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:49:10.966 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:10 smithi092 ceph-mon[62574]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:49:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:49:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:49:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:11.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:10 smithi159 ceph-mon[67110]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:49:11.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:49:11.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:49:11.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:11.999 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:11 smithi144 ceph-mon[67199]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:12.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:11 smithi092 ceph-mon[62574]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:12.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:11 smithi159 ceph-mon[67110]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:13.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:12 smithi159 ceph-mon[67110]: from='client.? 172.21.15.144:0/2201837107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e70e11e-c96c-48bb-916a-4ca295db76e1"}]: dispatch 2024-03-19T01:49:13.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:12 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e70e11e-c96c-48bb-916a-4ca295db76e1"}]: dispatch 2024-03-19T01:49:13.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:12 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e70e11e-c96c-48bb-916a-4ca295db76e1"}]': finished 2024-03-19T01:49:13.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:12 smithi159 ceph-mon[67110]: osdmap e69: 11 total, 10 up, 11 in 2024-03-19T01:49:13.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:13.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:12 smithi144 ceph-mon[67199]: from='client.? 172.21.15.144:0/2201837107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e70e11e-c96c-48bb-916a-4ca295db76e1"}]: dispatch 2024-03-19T01:49:13.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:12 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e70e11e-c96c-48bb-916a-4ca295db76e1"}]: dispatch 2024-03-19T01:49:13.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:12 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e70e11e-c96c-48bb-916a-4ca295db76e1"}]': finished 2024-03-19T01:49:13.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:12 smithi144 ceph-mon[67199]: osdmap e69: 11 total, 10 up, 11 in 2024-03-19T01:49:13.287 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:13.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:12 smithi092 ceph-mon[62574]: from='client.? 172.21.15.144:0/2201837107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e70e11e-c96c-48bb-916a-4ca295db76e1"}]: dispatch 2024-03-19T01:49:13.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:12 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e70e11e-c96c-48bb-916a-4ca295db76e1"}]: dispatch 2024-03-19T01:49:13.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:12 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e70e11e-c96c-48bb-916a-4ca295db76e1"}]': finished 2024-03-19T01:49:13.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:12 smithi092 ceph-mon[62574]: osdmap e69: 11 total, 10 up, 11 in 2024-03-19T01:49:13.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:14.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:13 smithi159 ceph-mon[67110]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:14.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:13 smithi159 ceph-mon[67110]: from='client.? 172.21.15.144:0/1082837478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:49:14.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:13 smithi092 ceph-mon[62574]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:14.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:13 smithi092 ceph-mon[62574]: from='client.? 172.21.15.144:0/1082837478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:49:14.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:13 smithi144 ceph-mon[67199]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:14.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:13 smithi144 ceph-mon[67199]: from='client.? 172.21.15.144:0/1082837478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:49:16.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:16 smithi092 ceph-mon[62574]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:16 smithi144 ceph-mon[67199]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:16 smithi159 ceph-mon[67110]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:18.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:18 smithi092 ceph-mon[62574]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:18.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:18 smithi144 ceph-mon[67199]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:18 smithi159 ceph-mon[67110]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:20.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:20 smithi144 ceph-mon[67199]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:20.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-19T01:49:20.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:20.757 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:20 smithi159 ceph-mon[67110]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:20.758 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-19T01:49:20.758 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:20 smithi092 ceph-mon[62574]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-19T01:49:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:21.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:21 smithi092 ceph-mon[62574]: Deploying daemon osd.10 on smithi144 2024-03-19T01:49:21.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:21 smithi144 ceph-mon[67199]: Deploying daemon osd.10 on smithi144 2024-03-19T01:49:21.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:21 smithi159 ceph-mon[67110]: Deploying daemon osd.10 on smithi144 2024-03-19T01:49:22.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:22 smithi144 ceph-mon[67199]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:22.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:22 smithi092 ceph-mon[62574]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:22 smithi159 ceph-mon[67110]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:24.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:24 smithi144 ceph-mon[67199]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:24.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:24 smithi092 ceph-mon[62574]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:24 smithi159 ceph-mon[67110]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:26.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:26 smithi092 ceph-mon[62574]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:26.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:26.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:26.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:26.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:26 smithi144 ceph-mon[67199]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:26.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:26.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:26.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:26 smithi159 ceph-mon[67110]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:28.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:28 smithi144 ceph-mon[67199]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:28 smithi092 ceph-mon[62574]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:28 smithi159 ceph-mon[67110]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:30.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.269+0000 7f782f7fe700 1 -- 172.21.15.144:0/1160272815 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7840062750 con 0x7f78280a2890 2024-03-19T01:49:30.269 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 10 on host 'smithi144' 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78280a2890 msgr2=0x7f78280a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78280a2890 0x7f78280a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f784006a810 tx=0x7f7834008040 comp rx=0 tx=0).stop 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103630 msgr2=0x7f7840198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103630 0x7f7840198af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f783000b530 tx=0x7f783000b840 comp rx=0 tx=0).stop 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 shutdown_connections 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78280a2890 0x7f78280a4d50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7840103fd0 0x7f784019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7840100650 0x7f78401985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 --2- 172.21.15.144:0/1160272815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7840103630 0x7f7840198af0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:30.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 >> 172.21.15.144:0/1160272815 conn(0x7f78400fbfb0 msgr2=0x7f7840106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:49:30.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 shutdown_connections 2024-03-19T01:49:30.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-19T01:49:30.271+0000 7f7844c8f700 1 -- 172.21.15.144:0/1160272815 wait complete. 2024-03-19T01:49:30.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:30.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:30.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:30.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='osd.10 [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:49:30.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:49:30.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:30.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:30.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:30.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:30.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='osd.10 [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:49:30.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:49:30.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:30.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='osd.10 [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:49:30.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-19T01:49:30.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:30.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:30.954 DEBUG:teuthology.orchestra.run.smithi144:osd.10> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.10.service 2024-03-19T01:49:30.956 INFO:tasks.cephadm:Deploying osd.11 on smithi159 with /dev/vg_nvme/lv_1... 2024-03-19T01:49:30.956 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-19T01:49:31.204 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:49:31.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:31 smithi144 ceph-mon[67199]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-19T01:49:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:31 smithi144 ceph-mon[67199]: osdmap e70: 11 total, 10 up, 11 in 2024-03-19T01:49:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:31 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:31 smithi144 ceph-mon[67199]: from='osd.10 [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:49:31.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:31 smithi144 ceph-mon[67199]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:49:31.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 01:49:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T01:49:31.443+0000 7fd6a693f700 -1 osd.10 0 waiting for initial osdmap 2024-03-19T01:49:31.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 01:49:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T01:49:31.454+0000 7fd6a2732700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:49:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:31 smithi159 ceph-mon[67110]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-19T01:49:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:31 smithi159 ceph-mon[67110]: osdmap e70: 11 total, 10 up, 11 in 2024-03-19T01:49:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:31 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:31 smithi159 ceph-mon[67110]: from='osd.10 [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:49:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:31 smithi159 ceph-mon[67110]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:49:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:31 smithi092 ceph-mon[62574]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-19T01:49:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:31 smithi092 ceph-mon[62574]: osdmap e70: 11 total, 10 up, 11 in 2024-03-19T01:49:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:31 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:31 smithi092 ceph-mon[62574]: from='osd.10 [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:49:31.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:31 smithi092 ceph-mon[62574]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-19T01:49:32.630 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:32 smithi159 ceph-mon[67110]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:32.630 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:32 smithi159 ceph-mon[67110]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:49:32.630 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:32 smithi159 ceph-mon[67110]: osdmap e71: 11 total, 10 up, 11 in 2024-03-19T01:49:32.631 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:32 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:32.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:32 smithi144 ceph-mon[67199]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:32.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:32 smithi144 ceph-mon[67199]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:49:32.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:32 smithi144 ceph-mon[67199]: osdmap e71: 11 total, 10 up, 11 in 2024-03-19T01:49:32.738 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:32 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:32 smithi092 ceph-mon[62574]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:32 smithi092 ceph-mon[62574]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-19T01:49:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:32 smithi092 ceph-mon[62574]: osdmap e71: 11 total, 10 up, 11 in 2024-03-19T01:49:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:32 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:33.477 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:33 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:49:33.477 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:33 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:49:33.477 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:33.477 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:33 smithi159 ceph-mon[67110]: osd.10 [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] boot 2024-03-19T01:49:33.477 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:33 smithi159 ceph-mon[67110]: osdmap e72: 11 total, 11 up, 11 in 2024-03-19T01:49:33.477 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:33.478 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-19T01:49:33.478 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-19T01:49:33.478 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-19T01:49:33.478 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-19T01:49:33.478 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114165 s, 918 MB/s 2024-03-19T01:49:33.478 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-19T01:49:33.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:33 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:49:33.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:33 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:49:33.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:33.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:33 smithi144 ceph-mon[67199]: osd.10 [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] boot 2024-03-19T01:49:33.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:33 smithi144 ceph-mon[67199]: osdmap e72: 11 total, 11 up, 11 in 2024-03-19T01:49:33.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:33.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:33 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:49:33.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:33 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:49:33.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:33.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:33 smithi092 ceph-mon[62574]: osd.10 [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] boot 2024-03-19T01:49:33.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:33 smithi092 ceph-mon[62574]: osdmap e72: 11 total, 11 up, 11 in 2024-03-19T01:49:33.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-19T01:49:34.176 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2024-03-19T01:49:34.419 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:49:34.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:34.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: osdmap e73: 11 total, 11 up, 11 in 2024-03-19T01:49:34.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.792 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.792 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.792 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.792 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.792 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.792 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:34.792 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:34.792 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:34 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: osdmap e73: 11 total, 11 up, 11 in 2024-03-19T01:49:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:34.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:34.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:34 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-19T01:49:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: osdmap e73: 11 total, 11 up, 11 in 2024-03-19T01:49:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:49:34.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:34.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:34.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:34.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:34 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:35.646 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:35 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi144 2024-03-19T01:49:35.646 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:35 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi144 to 4270M 2024-03-19T01:49:35.646 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:35 smithi159 ceph-mon[67110]: osdmap e74: 11 total, 11 up, 11 in 2024-03-19T01:49:35.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:35 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi144 2024-03-19T01:49:35.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:35 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi144 to 4270M 2024-03-19T01:49:35.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:35 smithi092 ceph-mon[62574]: osdmap e74: 11 total, 11 up, 11 in 2024-03-19T01:49:35.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:35 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi144 2024-03-19T01:49:35.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:35 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi144 to 4270M 2024-03-19T01:49:35.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:35 smithi144 ceph-mon[67199]: osdmap e74: 11 total, 11 up, 11 in 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 -- 172.21.15.159:0/341504333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103560 msgr2=0x7f11601039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 --2- 172.21.15.159:0/341504333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103560 0x7f11601039c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1148009990 tx=0x7f1148009ca0 comp rx=0 tx=0).stop 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 -- 172.21.15.159:0/341504333 shutdown_connections 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 --2- 172.21.15.159:0/341504333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1160103f00 0x7f116010a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 --2- 172.21.15.159:0/341504333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1160102360 0x7f1160102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 --2- 172.21.15.159:0/341504333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103560 0x7f11601039c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 -- 172.21.15.159:0/341504333 >> 172.21.15.159:0/341504333 conn(0x7f11600fd920 msgr2=0x7f11600ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 -- 172.21.15.159:0/341504333 shutdown_connections 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 -- 172.21.15.159:0/341504333 wait complete. 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 Processor -- start 2024-03-19T01:49:36.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.162+0000 7f1165da3700 1 -- start start 2024-03-19T01:49:36.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f1165da3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1160102360 0x7f11601985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:36.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f1165da3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1160103560 0x7f1160198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:36.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f1165da3700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103f00 0x7f116019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:36.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f1165da3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f116010dff0 con 0x7f1160103560 2024-03-19T01:49:36.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f1165da3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f116010de70 con 0x7f1160102360 2024-03-19T01:49:36.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f1165da3700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f116010e170 con 0x7f1160103f00 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103f00 0x7f116019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115ffff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103f00 0x7f116019fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60240/0 (socket says 172.21.15.159:60240) 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115ffff700 1 -- 172.21.15.159:0/2709674165 learned_addr learned my addr 172.21.15.159:0/2709674165 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f1156ffd700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1160103560 0x7f1160198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115f7fe700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1160102360 0x7f11601985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115ffff700 1 -- 172.21.15.159:0/2709674165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1160102360 msgr2=0x7f11601985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115ffff700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1160102360 0x7f11601985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115ffff700 1 -- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1160103560 msgr2=0x7f1160198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115ffff700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1160103560 0x7f1160198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115ffff700 1 -- 172.21.15.159:0/2709674165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11601a0200 con 0x7f1160103f00 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115ffff700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103f00 0x7f116019fb60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1158007f10 tx=0x7f115800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.163+0000 7f115d7fa700 1 -- 172.21.15.159:0/2709674165 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158004bb0 con 0x7f1160103f00 2024-03-19T01:49:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.164+0000 7f115d7fa700 1 -- 172.21.15.159:0/2709674165 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1158004d50 con 0x7f1160103f00 2024-03-19T01:49:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.164+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11601a04f0 con 0x7f1160103f00 2024-03-19T01:49:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.164+0000 7f115d7fa700 1 -- 172.21.15.159:0/2709674165 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11580059f0 con 0x7f1160103f00 2024-03-19T01:49:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.164+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11601a0ab0 con 0x7f1160103f00 2024-03-19T01:49:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.165+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1160068fd0 con 0x7f1160103f00 2024-03-19T01:49:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.166+0000 7f115d7fa700 1 -- 172.21.15.159:0/2709674165 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f115801a020 con 0x7f1160103f00 2024-03-19T01:49:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.166+0000 7f115d7fa700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f114c0a2890 0x7f114c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.167+0000 7f115d7fa700 1 -- 172.21.15.159:0/2709674165 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f11580cdb20 con 0x7f1160103f00 2024-03-19T01:49:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.167+0000 7f115f7fe700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f114c0a2890 0x7f114c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.167+0000 7f115f7fe700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f114c0a2890 0x7f114c0a4d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f11601033c0 tx=0x7f1150009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:49:36.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.171+0000 7f115d7fa700 1 -- 172.21.15.159:0/2709674165 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1158010070 con 0x7f1160103f00 2024-03-19T01:49:36.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:36.375+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1160062750 con 0x7f114c0a2890 2024-03-19T01:49:36.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:36 smithi159 ceph-mon[67110]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:36.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:36 smithi159 ceph-mon[67110]: osdmap e75: 11 total, 11 up, 11 in 2024-03-19T01:49:36.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:36 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:49:36.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:36 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:49:36.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:36 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:36 smithi092 ceph-mon[62574]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:36 smithi092 ceph-mon[62574]: osdmap e75: 11 total, 11 up, 11 in 2024-03-19T01:49:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:36 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:49:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:36 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:49:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:36 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:36 smithi144 ceph-mon[67199]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:36 smithi144 ceph-mon[67199]: osdmap e75: 11 total, 11 up, 11 in 2024-03-19T01:49:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:36 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-19T01:49:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:36 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-19T01:49:36.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:36 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:37.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:37 smithi159 ceph-mon[67110]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:49:37.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:37 smithi159 ceph-mon[67110]: osdmap e76: 11 total, 11 up, 11 in 2024-03-19T01:49:37.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:37 smithi092 ceph-mon[62574]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:49:37.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:37 smithi092 ceph-mon[62574]: osdmap e76: 11 total, 11 up, 11 in 2024-03-19T01:49:37.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:37 smithi144 ceph-mon[67199]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:49:37.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:37 smithi144 ceph-mon[67199]: osdmap e76: 11 total, 11 up, 11 in 2024-03-19T01:49:38.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:38 smithi159 ceph-mon[67110]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-19T01:49:38.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:38 smithi092 ceph-mon[62574]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-19T01:49:38.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:38 smithi144 ceph-mon[67199]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-19T01:49:39.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:39 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/4081730411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42fbc0b6-7285-451d-8c11-cb4179823811"}]: dispatch 2024-03-19T01:49:39.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:39 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42fbc0b6-7285-451d-8c11-cb4179823811"}]: dispatch 2024-03-19T01:49:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:39 smithi092 ceph-mon[62574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42fbc0b6-7285-451d-8c11-cb4179823811"}]': finished 2024-03-19T01:49:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:39 smithi092 ceph-mon[62574]: osdmap e77: 12 total, 11 up, 12 in 2024-03-19T01:49:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:39 smithi144 ceph-mon[67199]: from='client.? 172.21.15.159:0/4081730411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42fbc0b6-7285-451d-8c11-cb4179823811"}]: dispatch 2024-03-19T01:49:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:39 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42fbc0b6-7285-451d-8c11-cb4179823811"}]: dispatch 2024-03-19T01:49:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:39 smithi144 ceph-mon[67199]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42fbc0b6-7285-451d-8c11-cb4179823811"}]': finished 2024-03-19T01:49:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:39 smithi144 ceph-mon[67199]: osdmap e77: 12 total, 11 up, 12 in 2024-03-19T01:49:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:39 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/4081730411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42fbc0b6-7285-451d-8c11-cb4179823811"}]: dispatch 2024-03-19T01:49:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:39 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42fbc0b6-7285-451d-8c11-cb4179823811"}]: dispatch 2024-03-19T01:49:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:39 smithi159 ceph-mon[67110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42fbc0b6-7285-451d-8c11-cb4179823811"}]': finished 2024-03-19T01:49:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:39 smithi159 ceph-mon[67110]: osdmap e77: 12 total, 11 up, 12 in 2024-03-19T01:49:39.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:40 smithi092 ceph-mon[62574]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-19T01:49:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:40 smithi092 ceph-mon[62574]: from='client.? 172.21.15.159:0/1835559320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:49:40.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:40 smithi144 ceph-mon[67199]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-19T01:49:40.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:40 smithi144 ceph-mon[67199]: from='client.? 172.21.15.159:0/1835559320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:49:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:40 smithi159 ceph-mon[67110]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-19T01:49:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:40 smithi159 ceph-mon[67110]: from='client.? 172.21.15.159:0/1835559320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-19T01:49:42.786 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:42 smithi159 ceph-mon[67110]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T01:49:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:42 smithi092 ceph-mon[62574]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T01:49:42.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:42 smithi144 ceph-mon[67199]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-19T01:49:44.621 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:44 smithi159 ceph-mon[67110]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:49:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:44 smithi092 ceph-mon[62574]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:49:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:44 smithi144 ceph-mon[67199]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-19T01:49:46.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:46 smithi159 ceph-mon[67110]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:46 smithi092 ceph-mon[62574]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:46.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:46 smithi144 ceph-mon[67199]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:47.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-19T01:49:47.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-19T01:49:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-19T01:49:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:48.750 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:48 smithi159 ceph-mon[67110]: Deploying daemon osd.11 on smithi159 2024-03-19T01:49:48.750 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:48 smithi159 ceph-mon[67110]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:48.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:48 smithi092 ceph-mon[62574]: Deploying daemon osd.11 on smithi159 2024-03-19T01:49:48.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:48 smithi092 ceph-mon[62574]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:48.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:48 smithi144 ceph-mon[67199]: Deploying daemon osd.11 on smithi159 2024-03-19T01:49:48.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:48 smithi144 ceph-mon[67199]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:50.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:50 smithi159 ceph-mon[67110]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:50 smithi092 ceph-mon[62574]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:50.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:50 smithi144 ceph-mon[67199]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:52.542 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:52 smithi159 ceph-mon[67110]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:52 smithi092 ceph-mon[62574]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:52.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:52 smithi144 ceph-mon[67199]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:54.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:53 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:54.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:53 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:54.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:53 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:54.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:53 smithi159 ceph-mon[67110]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:54.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:53 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:54.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:53 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:54.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:53 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:54.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:53 smithi092 ceph-mon[62574]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:54.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:53 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:54.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:53 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:54.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:53 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:54.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:53 smithi144 ceph-mon[67199]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:56.586 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:56 smithi159 ceph-mon[67110]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:56.586 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:56.586 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:56.586 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:56.587 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:56.587 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:56 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:56.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:56 smithi092 ceph-mon[62574]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:56.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:56.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:56.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:56.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:56.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:56 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:56.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:56 smithi144 ceph-mon[67199]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:56.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:56.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:56.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:49:56.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:49:56.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:56 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:56.941 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 11 on host 'smithi159' 2024-03-19T01:49:56.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.940+0000 7f115d7fa700 1 -- 172.21.15.159:0/2709674165 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1160062750 con 0x7f114c0a2890 2024-03-19T01:49:56.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.942+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f114c0a2890 msgr2=0x7f114c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:56.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.942+0000 7f1165da3700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f114c0a2890 0x7f114c0a4d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f11601033c0 tx=0x7f1150009190 comp rx=0 tx=0).stop 2024-03-19T01:49:56.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103f00 msgr2=0x7f116019fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:56.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103f00 0x7f116019fb60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1158007f10 tx=0x7f115800d3b0 comp rx=0 tx=0).stop 2024-03-19T01:49:56.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 shutdown_connections 2024-03-19T01:49:56.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f114c0a2890 0x7f114c0a4d50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:56.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1160103f00 0x7f116019fb60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:56.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1160102360 0x7f11601985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:56.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 --2- 172.21.15.159:0/2709674165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1160103560 0x7f1160198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:56.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 >> 172.21.15.159:0/2709674165 conn(0x7f11600fd920 msgr2=0x7f1160106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:49:56.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 shutdown_connections 2024-03-19T01:49:56.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:49:56.943+0000 7f1165da3700 1 -- 172.21.15.159:0/2709674165 wait complete. 2024-03-19T01:49:57.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:57 smithi159 ceph-mon[67110]: from='osd.11 [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:49:57.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:57 smithi159 ceph-mon[67110]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:49:57.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:57 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:57.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:57 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:57.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:57 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:57.761 DEBUG:teuthology.orchestra.run.smithi159:osd.11> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.11.service 2024-03-19T01:49:57.763 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-19T01:49:57.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-19T01:49:57.793 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:57 smithi092 ceph-mon[62574]: from='osd.11 [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:49:57.793 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:57 smithi092 ceph-mon[62574]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:49:57.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:57 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:57.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:57 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:57.794 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:57 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:57.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:57 smithi144 ceph-mon[67199]: from='osd.11 [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:49:57.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:57 smithi144 ceph-mon[67199]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-19T01:49:57.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:57 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:57.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:57 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:49:57.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:57 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:49:58.014 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:49:58.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:58 smithi159 ceph-mon[67110]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:58.517 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:58 smithi159 ceph-mon[67110]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-19T01:49:58.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:58 smithi159 ceph-mon[67110]: osdmap e78: 12 total, 11 up, 12 in 2024-03-19T01:49:58.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:58 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:58.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:58 smithi159 ceph-mon[67110]: from='osd.11 [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:49:58.518 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:58 smithi159 ceph-mon[67110]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:49:58.518 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 01:49:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T01:49:58.504+0000 7fc9d60b0700 -1 osd.11 0 waiting for initial osdmap 2024-03-19T01:49:58.518 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 01:49:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T01:49:58.517+0000 7fc9d1ea3700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-19T01:49:58.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:58 smithi092 ceph-mon[62574]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:58.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:58 smithi092 ceph-mon[62574]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-19T01:49:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:58 smithi092 ceph-mon[62574]: osdmap e78: 12 total, 11 up, 12 in 2024-03-19T01:49:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:58 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:58 smithi092 ceph-mon[62574]: from='osd.11 [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:49:58.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:58 smithi092 ceph-mon[62574]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:49:58.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:58 smithi144 ceph-mon[67199]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:49:58.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:58 smithi144 ceph-mon[67199]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-19T01:49:58.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:58 smithi144 ceph-mon[67199]: osdmap e78: 12 total, 11 up, 12 in 2024-03-19T01:49:58.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:58 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:58.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:58 smithi144 ceph-mon[67199]: from='osd.11 [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:49:58.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:58 smithi144 ceph-mon[67199]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-19T01:49:59.655 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:59 smithi159 ceph-mon[67110]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:49:59.655 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:59 smithi159 ceph-mon[67110]: osdmap e79: 12 total, 11 up, 12 in 2024-03-19T01:49:59.656 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:59 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:59.656 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:49:59 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:59.697 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:59 smithi092 ceph-mon[62574]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:49:59.697 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:59 smithi092 ceph-mon[62574]: osdmap e79: 12 total, 11 up, 12 in 2024-03-19T01:49:59.697 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:59 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:59.697 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:49:59 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.798+0000 7f585cdd1700 1 -- 172.21.15.92:0/3864996855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 msgr2=0x7f58581039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.798+0000 7f585cdd1700 1 --2- 172.21.15.92:0/3864996855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 0x7f58581039c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f584c0099c0 tx=0x7f584c009cd0 comp rx=0 tx=0).stop 2024-03-19T01:49:59.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.798+0000 7f585cdd1700 1 -- 172.21.15.92:0/3864996855 shutdown_connections 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.798+0000 7f585cdd1700 1 --2- 172.21.15.92:0/3864996855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5858103f00 0x7f585810a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.798+0000 7f585cdd1700 1 --2- 172.21.15.92:0/3864996855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5858102360 0x7f5858102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.798+0000 7f585cdd1700 1 --2- 172.21.15.92:0/3864996855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 0x7f58581039c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.798+0000 7f585cdd1700 1 -- 172.21.15.92:0/3864996855 >> 172.21.15.92:0/3864996855 conn(0x7f58580fd920 msgr2=0x7f58580ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.798+0000 7f585cdd1700 1 -- 172.21.15.92:0/3864996855 shutdown_connections 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.798+0000 7f585cdd1700 1 -- 172.21.15.92:0/3864996855 wait complete. 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f585cdd1700 1 Processor -- start 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f585cdd1700 1 -- start start 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f585cdd1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5858102360 0x7f58581985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f585cdd1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 0x7f5858198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f585cdd1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5858103f00 0x7f585819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:59.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f585cdd1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f585810dfc0 con 0x7f5858103560 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f585cdd1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f585810de40 con 0x7f5858103f00 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f585cdd1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f585810e140 con 0x7f5858102360 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f5855d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 0x7f5858198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f5855d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 0x7f5858198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58638/0 (socket says 172.21.15.92:58638) 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f5855d9b700 1 -- 172.21.15.92:0/2993279707 learned_addr learned my addr 172.21.15.92:0/2993279707 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.799+0000 7f585659c700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5858102360 0x7f58581985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f5856d9d700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5858103f00 0x7f585819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f5855d9b700 1 -- 172.21.15.92:0/2993279707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5858102360 msgr2=0x7f58581985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f5855d9b700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5858102360 0x7f58581985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f5855d9b700 1 -- 172.21.15.92:0/2993279707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5858103f00 msgr2=0x7f585819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f5855d9b700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5858103f00 0x7f585819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f5855d9b700 1 -- 172.21.15.92:0/2993279707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58581a0230 con 0x7f5858103560 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f5856d9d700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5858103f00 0x7f585819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:49:59.807 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f5855d9b700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 0x7f5858198b10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f584c00b580 tx=0x7f584c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f58477fe700 1 -- 172.21.15.92:0/2993279707 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c027070 con 0x7f5858103560 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58581a04c0 con 0x7f5858103560 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58581a09f0 con 0x7f5858103560 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f58477fe700 1 -- 172.21.15.92:0/2993279707 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584c004150 con 0x7f5858103560 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.800+0000 7f58477fe700 1 -- 172.21.15.92:0/2993279707 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c022e60 con 0x7f5858103560 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.801+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5858068fd0 con 0x7f5858103560 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.802+0000 7f58477fe700 1 -- 172.21.15.92:0/2993279707 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f584c02a070 con 0x7f5858103560 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.802+0000 7f58477fe700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f58400a2890 0x7f58400a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.803+0000 7f585659c700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f58400a2890 0x7f58400a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.803+0000 7f58477fe700 1 -- 172.21.15.92:0/2993279707 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f584c0cf500 con 0x7f5858103560 2024-03-19T01:49:59.808 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.803+0000 7f585659c700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f58400a2890 0x7f58400a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f58581033c0 tx=0x7f5848009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:49:59.809 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:49:59.806+0000 7f58477fe700 1 -- 172.21.15.92:0/2993279707 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f584c013070 con 0x7f5858103560 2024-03-19T01:49:59.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:59 smithi144 ceph-mon[67199]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-19T01:49:59.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:59 smithi144 ceph-mon[67199]: osdmap e79: 12 total, 11 up, 12 in 2024-03-19T01:49:59.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:59 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:49:59.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:49:59 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:50:00.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.000+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f585802dbb0 con 0x7f5858103560 2024-03-19T01:50:00.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.001+0000 7f58477fe700 1 -- 172.21.15.92:0/2993279707 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f584c09cad0 con 0x7f5858103560 2024-03-19T01:50:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:00.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f58400a2890 msgr2=0x7f58400a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:00.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f58400a2890 0x7f58400a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f58581033c0 tx=0x7f5848009190 comp rx=0 tx=0).stop 2024-03-19T01:50:00.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 msgr2=0x7f5858198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:00.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 0x7f5858198b10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f584c00b580 tx=0x7f584c009580 comp rx=0 tx=0).stop 2024-03-19T01:50:00.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 shutdown_connections 2024-03-19T01:50:00.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f58400a2890 0x7f58400a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:00.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5858103f00 0x7f585819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:00.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5858102360 0x7f58581985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:00.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 --2- 172.21.15.92:0/2993279707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858103560 0x7f5858198b10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:00.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 >> 172.21.15.92:0/2993279707 conn(0x7f58580fd920 msgr2=0x7f58581068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:00.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 shutdown_connections 2024-03-19T01:50:00.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:00.003+0000 7f585cdd1700 1 -- 172.21.15.92:0/2993279707 wait complete. 2024-03-19T01:50:00.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:00 smithi092 ceph-mon[62574]: purged_snaps scrub starts 2024-03-19T01:50:00.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:00 smithi092 ceph-mon[62574]: purged_snaps scrub ok 2024-03-19T01:50:00.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:00 smithi092 ceph-mon[62574]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:50:00.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:00 smithi092 ceph-mon[62574]: osd.11 [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] boot 2024-03-19T01:50:00.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:00 smithi092 ceph-mon[62574]: osdmap e80: 12 total, 12 up, 12 in 2024-03-19T01:50:00.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:00 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:50:00.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:00 smithi092 ceph-mon[62574]: overall HEALTH_OK 2024-03-19T01:50:00.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:00 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2993279707' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T01:50:00.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:00 smithi159 ceph-mon[67110]: purged_snaps scrub starts 2024-03-19T01:50:00.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:00 smithi159 ceph-mon[67110]: purged_snaps scrub ok 2024-03-19T01:50:00.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:00 smithi159 ceph-mon[67110]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:50:00.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:00 smithi159 ceph-mon[67110]: osd.11 [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] boot 2024-03-19T01:50:00.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:00 smithi159 ceph-mon[67110]: osdmap e80: 12 total, 12 up, 12 in 2024-03-19T01:50:00.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:00 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:50:00.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:00 smithi159 ceph-mon[67110]: overall HEALTH_OK 2024-03-19T01:50:00.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:00 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2993279707' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T01:50:00.632 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1710812999,"num_in_osds":12,"osd_in_since":1710812979,"num_remapped_pgs":0} 2024-03-19T01:50:00.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-19T01:50:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:00 smithi144 ceph-mon[67199]: purged_snaps scrub starts 2024-03-19T01:50:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:00 smithi144 ceph-mon[67199]: purged_snaps scrub ok 2024-03-19T01:50:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:00 smithi144 ceph-mon[67199]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-19T01:50:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:00 smithi144 ceph-mon[67199]: osd.11 [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] boot 2024-03-19T01:50:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:00 smithi144 ceph-mon[67199]: osdmap e80: 12 total, 12 up, 12 in 2024-03-19T01:50:00.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:00 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-19T01:50:00.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:00 smithi144 ceph-mon[67199]: overall HEALTH_OK 2024-03-19T01:50:00.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:00 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2993279707' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-19T01:50:00.915 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:50:01.744 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: osdmap e81: 12 total, 12 up, 12 in 2024-03-19T01:50:01.744 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:01.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:01.745 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:01 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: osdmap e81: 12 total, 12 up, 12 in 2024-03-19T01:50:01.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:01.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:01.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:01 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: osdmap e81: 12 total, 12 up, 12 in 2024-03-19T01:50:01.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:50:01.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:01.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:01.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:01.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:01 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:02.608 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:02 smithi092 ceph-mon[62574]: Detected new or changed devices on smithi159 2024-03-19T01:50:02.608 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:02 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi159 to 4270M 2024-03-19T01:50:02.608 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:02 smithi092 ceph-mon[62574]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:02.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.609+0000 7fd984bd1700 1 -- 172.21.15.92:0/3206014773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd980074540 msgr2=0x7fd980074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:02.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.609+0000 7fd984bd1700 1 --2- 172.21.15.92:0/3206014773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd980074540 0x7fd980074920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd970009990 tx=0x7fd970009ca0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.609+0000 7fd984bd1700 1 -- 172.21.15.92:0/3206014773 shutdown_connections 2024-03-19T01:50:02.609 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.609+0000 7fd984bd1700 1 --2- 172.21.15.92:0/3206014773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9801134e0 0x7fd9801158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.609+0000 7fd984bd1700 1 --2- 172.21.15.92:0/3206014773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd980074540 0x7fd980074920 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.609+0000 7fd984bd1700 1 --2- 172.21.15.92:0/3206014773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd980074e60 0x7fd980112fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.609+0000 7fd984bd1700 1 -- 172.21.15.92:0/3206014773 >> 172.21.15.92:0/3206014773 conn(0x7fd980079440 msgr2=0x7fd980079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:02.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.609+0000 7fd984bd1700 1 -- 172.21.15.92:0/3206014773 shutdown_connections 2024-03-19T01:50:02.610 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.609+0000 7fd984bd1700 1 -- 172.21.15.92:0/3206014773 wait complete. 2024-03-19T01:50:02.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd984bd1700 1 Processor -- start 2024-03-19T01:50:02.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd984bd1700 1 -- start start 2024-03-19T01:50:02.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd984bd1700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd980074540 0x7fd98019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd984bd1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980074e60 0x7fd98019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd984bd1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9801134e0 0x7fd9801a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd984bd1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd980118d40 con 0x7fd9801134e0 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd984bd1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd980118bc0 con 0x7fd980074e60 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd984bd1700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd980118ec0 con 0x7fd980074540 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd97ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9801134e0 0x7fd9801a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd97ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9801134e0 0x7fd9801a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58662/0 (socket says 172.21.15.92:58662) 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd97ed9d700 1 -- 172.21.15.92:0/7496156 learned_addr learned my addr 172.21.15.92:0/7496156 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd97e59c700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd980074540 0x7fd98019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.610+0000 7fd977fff700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980074e60 0x7fd98019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:02.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd97ed9d700 1 -- 172.21.15.92:0/7496156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd980074540 msgr2=0x7fd98019f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd97ed9d700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd980074540 0x7fd98019f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd97ed9d700 1 -- 172.21.15.92:0/7496156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980074e60 msgr2=0x7fd98019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd97ed9d700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980074e60 0x7fd98019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd97ed9d700 1 -- 172.21.15.92:0/7496156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9801a4200 con 0x7fd9801134e0 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd977fff700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980074e60 0x7fd98019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd97ed9d700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9801134e0 0x7fd9801a3ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd97800b9e0 tx=0x7fd97800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd9777fe700 1 -- 172.21.15.92:0/7496156 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978019070 con 0x7fd9801134e0 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9801a44f0 con 0x7fd9801134e0 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd9777fe700 1 -- 172.21.15.92:0/7496156 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd97800ce10 con 0x7fd9801134e0 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd9777fe700 1 -- 172.21.15.92:0/7496156 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978012360 con 0x7fd9801134e0 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.611+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9801abd70 con 0x7fd9801134e0 2024-03-19T01:50:02.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.612+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd980068fd0 con 0x7fd9801134e0 2024-03-19T01:50:02.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.613+0000 7fd9777fe700 1 -- 172.21.15.92:0/7496156 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd97801c070 con 0x7fd9801134e0 2024-03-19T01:50:02.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.614+0000 7fd9777fe700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd96c0a2890 0x7fd96c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:02.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.614+0000 7fd97e59c700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd96c0a2890 0x7fd96c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:02.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.614+0000 7fd9777fe700 1 -- 172.21.15.92:0/7496156 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd9780cdb60 con 0x7fd9801134e0 2024-03-19T01:50:02.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.614+0000 7fd97e59c700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd96c0a2890 0x7fd96c0a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b5c0 tx=0x7fd970005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:02.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.617+0000 7fd9777fe700 1 -- 172.21.15.92:0/7496156 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd97809b0a0 con 0x7fd9801134e0 2024-03-19T01:50:02.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.813+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9801abfa0 con 0x7fd9801134e0 2024-03-19T01:50:02.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.814+0000 7fd9777fe700 1 -- 172.21.15.92:0/7496156 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18493 (secure 0 0 0) 0x7fd97809a9a0 con 0x7fd9801134e0 2024-03-19T01:50:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:02.815 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":81,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:50:00.507820+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd96c0a2890 msgr2=0x7fd96c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd96c0a2890 0x7fd96c0a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b5c0 tx=0x7fd970005960 comp rx=0 tx=0).stop 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9801134e0 msgr2=0x7fd9801a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9801134e0 0x7fd9801a3ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd97800b9e0 tx=0x7fd97800bcf0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 shutdown_connections 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd96c0a2890 0x7fd96c0a4d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9801134e0 0x7fd9801a3ac0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd980074540 0x7fd98019f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 --2- 172.21.15.92:0/7496156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd980074e60 0x7fd98019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.816+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 >> 172.21.15.92:0/7496156 conn(0x7fd980079440 msgr2=0x7fd9801122b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.817+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 shutdown_connections 2024-03-19T01:50:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:02.817+0000 7fd984bd1700 1 -- 172.21.15.92:0/7496156 wait complete. 2024-03-19T01:50:02.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:02 smithi144 ceph-mon[67199]: Detected new or changed devices on smithi159 2024-03-19T01:50:02.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:02 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi159 to 4270M 2024-03-19T01:50:02.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:02 smithi144 ceph-mon[67199]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:02 smithi159 ceph-mon[67110]: Detected new or changed devices on smithi159 2024-03-19T01:50:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:02 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi159 to 4270M 2024-03-19T01:50:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:02 smithi159 ceph-mon[67110]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:03.583 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:46:07.695753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:50:03.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-19T01:50:03.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:03 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/7496156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:03.820 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:50:03.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:03 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/7496156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:03.880 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:03 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/7496156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:04.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:04 smithi092 ceph-mon[62574]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:04.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:04 smithi144 ceph-mon[67199]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:04 smithi159 ceph-mon[67110]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.655+0000 7f231647e700 1 -- 172.21.15.92:0/4052418280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006aa60 msgr2=0x7f231006aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.655+0000 7f231647e700 1 --2- 172.21.15.92:0/4052418280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006aa60 0x7f231006aee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f23040099c0 tx=0x7f2304009cd0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.655+0000 7f231647e700 1 -- 172.21.15.92:0/4052418280 shutdown_connections 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.655+0000 7f231647e700 1 --2- 172.21.15.92:0/4052418280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f231006b420 0x7f231010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.655+0000 7f231647e700 1 --2- 172.21.15.92:0/4052418280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2310108340 0x7f2310108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.655+0000 7f231647e700 1 --2- 172.21.15.92:0/4052418280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006aa60 0x7f231006aee0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.655+0000 7f231647e700 1 -- 172.21.15.92:0/4052418280 >> 172.21.15.92:0/4052418280 conn(0x7f2310078220 msgr2=0x7f2310078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.655+0000 7f231647e700 1 -- 172.21.15.92:0/4052418280 shutdown_connections 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.655+0000 7f231647e700 1 -- 172.21.15.92:0/4052418280 wait complete. 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f231647e700 1 Processor -- start 2024-03-19T01:50:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f231647e700 1 -- start start 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f231647e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f231006aa60 0x7f231019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f231647e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006b420 0x7f231019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f231647e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2310108340 0x7f2310198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f231647e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23101125f0 con 0x7f231006b420 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f231647e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2310112470 con 0x7f231006aa60 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f231647e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2310112770 con 0x7f2310108340 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f230f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006b420 0x7f231019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f230f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006b420 0x7f231019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51248/0 (socket says 172.21.15.92:51248) 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f230f7fe700 1 -- 172.21.15.92:0/179529204 learned_addr learned my addr 172.21.15.92:0/179529204 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.656+0000 7f2314a1b700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2310108340 0x7f2310198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230ffff700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f231006aa60 0x7f231019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230f7fe700 1 -- 172.21.15.92:0/179529204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2310108340 msgr2=0x7f2310198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230f7fe700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2310108340 0x7f2310198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230f7fe700 1 -- 172.21.15.92:0/179529204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f231006aa60 msgr2=0x7f231019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230f7fe700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f231006aa60 0x7f231019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230f7fe700 1 -- 172.21.15.92:0/179529204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2310199030 con 0x7f231006b420 2024-03-19T01:50:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230f7fe700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006b420 0x7f231019ec10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2304006010 tx=0x7f2304009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230d7fa700 1 -- 172.21.15.92:0/179529204 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304027070 con 0x7f231006b420 2024-03-19T01:50:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23101992c0 con 0x7f231006b420 2024-03-19T01:50:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230d7fa700 1 -- 172.21.15.92:0/179529204 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2304003780 con 0x7f231006b420 2024-03-19T01:50:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f230d7fa700 1 -- 172.21.15.92:0/179529204 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304022da0 con 0x7f231006b420 2024-03-19T01:50:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.657+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23101a5530 con 0x7f231006b420 2024-03-19T01:50:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.658+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23101039f0 con 0x7f231006b420 2024-03-19T01:50:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.659+0000 7f230d7fa700 1 -- 172.21.15.92:0/179529204 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f230402a070 con 0x7f231006b420 2024-03-19T01:50:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.660+0000 7f230d7fa700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22f80a2890 0x7f22f80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.660+0000 7f230ffff700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22f80a2890 0x7f22f80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.660+0000 7f230d7fa700 1 -- 172.21.15.92:0/179529204 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f23040cf3d0 con 0x7f231006b420 2024-03-19T01:50:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.660+0000 7f230ffff700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22f80a2890 0x7f22f80a4d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2300005fd0 tx=0x7f2300005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.664+0000 7f230d7fa700 1 -- 172.21.15.92:0/179529204 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f230405e740 con 0x7f231006b420 2024-03-19T01:50:05.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.858+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2310199d80 con 0x7f231006b420 2024-03-19T01:50:05.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.858+0000 7f230d7fa700 1 -- 172.21.15.92:0/179529204 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f2304013070 con 0x7f231006b420 2024-03-19T01:50:05.859 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.860+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22f80a2890 msgr2=0x7f22f80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.860+0000 7f231647e700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22f80a2890 0x7f22f80a4d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2300005fd0 tx=0x7f2300005db0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.860+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006b420 msgr2=0x7f231019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.860+0000 7f231647e700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006b420 0x7f231019ec10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2304006010 tx=0x7f2304009f80 comp rx=0 tx=0).stop 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.860+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 shutdown_connections 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.860+0000 7f231647e700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22f80a2890 0x7f22f80a4d50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.860+0000 7f231647e700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2310108340 0x7f2310198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.861+0000 7f231647e700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f231006aa60 0x7f231019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.861+0000 7f231647e700 1 --2- 172.21.15.92:0/179529204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f231006b420 0x7f231019ec10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.861+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 >> 172.21.15.92:0/179529204 conn(0x7f2310078220 msgr2=0x7f23100ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:05.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.861+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 shutdown_connections 2024-03-19T01:50:05.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:05.861+0000 7f231647e700 1 -- 172.21.15.92:0/179529204 wait complete. 2024-03-19T01:50:06.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:06 smithi092 ceph-mon[62574]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:06.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:06 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/179529204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:50:06.687 INFO:tasks.cephadm:Adding mds.a on smithi092 2024-03-19T01:50:06.687 INFO:tasks.cephadm:Adding mds.d on smithi092 2024-03-19T01:50:06.687 INFO:tasks.cephadm:Adding mds.g on smithi092 2024-03-19T01:50:06.687 INFO:tasks.cephadm:Adding mds.j on smithi092 2024-03-19T01:50:06.687 INFO:tasks.cephadm:Adding mds.b on smithi144 2024-03-19T01:50:06.687 INFO:tasks.cephadm:Adding mds.e on smithi144 2024-03-19T01:50:06.688 INFO:tasks.cephadm:Adding mds.h on smithi144 2024-03-19T01:50:06.688 INFO:tasks.cephadm:Adding mds.k on smithi144 2024-03-19T01:50:06.688 INFO:tasks.cephadm:Adding mds.c on smithi159 2024-03-19T01:50:06.688 INFO:tasks.cephadm:Adding mds.f on smithi159 2024-03-19T01:50:06.688 INFO:tasks.cephadm:Adding mds.i on smithi159 2024-03-19T01:50:06.688 INFO:tasks.cephadm:Adding mds.l on smithi159 2024-03-19T01:50:06.688 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l' 2024-03-19T01:50:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:06 smithi144 ceph-mon[67199]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:06 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/179529204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:50:06.939 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T01:50:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:06 smithi159 ceph-mon[67110]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:06 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/179529204' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:50:08.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.669+0000 7f987e53b700 1 -- 172.21.15.159:0/946879421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98781028a0 msgr2=0x7f9878102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:08.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.669+0000 7f987e53b700 1 --2- 172.21.15.159:0/946879421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98781028a0 0x7f9878102cc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9868009990 tx=0x7f9868009ca0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.669+0000 7f987e53b700 1 -- 172.21.15.159:0/946879421 shutdown_connections 2024-03-19T01:50:08.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.669+0000 7f987e53b700 1 --2- 172.21.15.159:0/946879421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9878101ec0 0x7f9878109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.669+0000 7f987e53b700 1 --2- 172.21.15.159:0/946879421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98781028a0 0x7f9878102cc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.669+0000 7f987e53b700 1 --2- 172.21.15.159:0/946879421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878101500 0x7f9878101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.669+0000 7f987e53b700 1 -- 172.21.15.159:0/946879421 >> 172.21.15.159:0/946879421 conn(0x7f98780fbfb0 msgr2=0x7f98780fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:08.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.669+0000 7f987e53b700 1 -- 172.21.15.159:0/946879421 shutdown_connections 2024-03-19T01:50:08.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.669+0000 7f987e53b700 1 -- 172.21.15.159:0/946879421 wait complete. 2024-03-19T01:50:08.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f987e53b700 1 Processor -- start 2024-03-19T01:50:08.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f987e53b700 1 -- start start 2024-03-19T01:50:08.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f987e53b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9878101500 0x7f9878198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:08.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f987e53b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878101ec0 0x7f9878198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:08.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f987e53b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98781028a0 0x7f987819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:08.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f987e53b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f987810cf40 con 0x7f9878101ec0 2024-03-19T01:50:08.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f987e53b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f987810cdc0 con 0x7f98781028a0 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f987e53b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f987810d0c0 con 0x7f9878101500 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f9877fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9878101500 0x7f9878198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f9877fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9878101500 0x7f9878198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58592/0 (socket says 172.21.15.159:58592) 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.670+0000 7f9877fff700 1 -- 172.21.15.159:0/3544360950 learned_addr learned my addr 172.21.15.159:0/3544360950 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f98777fe700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878101ec0 0x7f9878198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f987cad8700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98781028a0 0x7f987819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f9877fff700 1 -- 172.21.15.159:0/3544360950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98781028a0 msgr2=0x7f987819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f9877fff700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98781028a0 0x7f987819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f9877fff700 1 -- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878101ec0 msgr2=0x7f9878198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f9877fff700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878101ec0 0x7f9878198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f9877fff700 1 -- 172.21.15.159:0/3544360950 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98781a0380 con 0x7f9878101500 2024-03-19T01:50:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f98777fe700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878101ec0 0x7f9878198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:50:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f9877fff700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9878101500 0x7f9878198720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9868000c00 tx=0x7f9868009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f98757fa700 1 -- 172.21.15.159:0/3544360950 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868027070 con 0x7f9878101500 2024-03-19T01:50:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f98757fa700 1 -- 172.21.15.159:0/3544360950 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9868004150 con 0x7f9878101500 2024-03-19T01:50:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98781a0610 con 0x7f9878101500 2024-03-19T01:50:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f98757fa700 1 -- 172.21.15.159:0/3544360950 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986800f6c0 con 0x7f9878101500 2024-03-19T01:50:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.671+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98781a0ae0 con 0x7f9878101500 2024-03-19T01:50:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.672+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9878069000 con 0x7f9878101500 2024-03-19T01:50:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.673+0000 7f98757fa700 1 -- 172.21.15.159:0/3544360950 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f986802a070 con 0x7f9878101500 2024-03-19T01:50:08.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.674+0000 7f98757fa700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f98640a2890 0x7f98640a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:08.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.674+0000 7f98757fa700 1 -- 172.21.15.159:0/3544360950 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f98680cfe60 con 0x7f9878101500 2024-03-19T01:50:08.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.674+0000 7f98777fe700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f98640a2890 0x7f98640a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:08.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.674+0000 7f98777fe700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f98640a2890 0x7f98640a4d50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9860007900 tx=0x7f9860008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:08.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.677+0000 7f98757fa700 1 -- 172.21.15.159:0/3544360950 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9878069000 con 0x7f9878101500 2024-03-19T01:50:08.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:08 smithi159 ceph-mon[67110]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:08.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:08 smithi092 ceph-mon[62574]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:08.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:08 smithi144 ceph-mon[67199]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:08.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.886+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}) -- 0x7f98780fe2d0 con 0x7f98640a2890 2024-03-19T01:50:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.895+0000 7f98757fa700 1 -- 172.21.15.159:0/3544360950 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f98780fe2d0 con 0x7f98640a2890 2024-03-19T01:50:08.895 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mds.all update... 2024-03-19T01:50:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f98640a2890 msgr2=0x7f98640a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f98640a2890 0x7f98640a4d50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9860007900 tx=0x7f9860008040 comp rx=0 tx=0).stop 2024-03-19T01:50:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9878101500 msgr2=0x7f9878198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9878101500 0x7f9878198720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9868000c00 tx=0x7f9868009ee0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 shutdown_connections 2024-03-19T01:50:08.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f98640a2890 0x7f98640a4d50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98781028a0 0x7f987819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9878101500 0x7f9878198720 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 --2- 172.21.15.159:0/3544360950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878101ec0 0x7f9878198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:08.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 >> 172.21.15.159:0/3544360950 conn(0x7f98780fbfb0 msgr2=0x7f98780fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:08.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 shutdown_connections 2024-03-19T01:50:08.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-19T01:50:08.897+0000 7f987e53b700 1 -- 172.21.15.159:0/3544360950 wait complete. 2024-03-19T01:50:09.598 DEBUG:teuthology.orchestra.run.smithi092:mds.a> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.a.service 2024-03-19T01:50:09.601 DEBUG:teuthology.orchestra.run.smithi092:mds.d> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.d.service 2024-03-19T01:50:09.644 DEBUG:teuthology.orchestra.run.smithi092:mds.g> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.g.service 2024-03-19T01:50:09.687 DEBUG:teuthology.orchestra.run.smithi092:mds.j> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.j.service 2024-03-19T01:50:09.730 DEBUG:teuthology.orchestra.run.smithi144:mds.b> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.b.service 2024-03-19T01:50:09.732 DEBUG:teuthology.orchestra.run.smithi144:mds.e> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.e.service 2024-03-19T01:50:09.775 DEBUG:teuthology.orchestra.run.smithi144:mds.h> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.h.service 2024-03-19T01:50:09.818 DEBUG:teuthology.orchestra.run.smithi144:mds.k> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.k.service 2024-03-19T01:50:09.861 DEBUG:teuthology.orchestra.run.smithi159:mds.c> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.c.service 2024-03-19T01:50:09.863 DEBUG:teuthology.orchestra.run.smithi159:mds.f> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.f.service 2024-03-19T01:50:09.906 DEBUG:teuthology.orchestra.run.smithi159:mds.i> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.i.service 2024-03-19T01:50:09.949 DEBUG:teuthology.orchestra.run.smithi159:mds.l> sudo journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.l.service 2024-03-19T01:50:09.992 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-19T01:50:09.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:50:10.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:10.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: Saving service mds.all spec with placement smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-03-19T01:50:10.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:10.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:50:10.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:10.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:10.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:10.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:10.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:10.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:10.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:09 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 -- 172.21.15.92:0/552182191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 msgr2=0x7efd7c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 --2- 172.21.15.92:0/552182191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 0x7efd7c108b80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efd74009ca0 tx=0x7efd7402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 -- 172.21.15.92:0/552182191 shutdown_connections 2024-03-19T01:50:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 --2- 172.21.15.92:0/552182191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 0x7efd7c108b80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 --2- 172.21.15.92:0/552182191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd7c101930 0x7efd7c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 --2- 172.21.15.92:0/552182191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd7c100740 0x7efd7c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 -- 172.21.15.92:0/552182191 >> 172.21.15.92:0/552182191 conn(0x7efd7c0fbcb0 msgr2=0x7efd7c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 -- 172.21.15.92:0/552182191 shutdown_connections 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 -- 172.21.15.92:0/552182191 wait complete. 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.100+0000 7efd814a1640 1 Processor -- start 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd814a1640 1 -- start start 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd814a1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd7c100740 0x7efd7c1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd814a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd7c101930 0x7efd7c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd814a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 0x7efd7c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd814a1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c10c3a0 con 0x7efd7c1022f0 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd814a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c10c220 con 0x7efd7c101930 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd814a1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c10c520 con 0x7efd7c100740 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 0x7efd7c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 0x7efd7c19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51264/0 (socket says 172.21.15.92:51264) 2024-03-19T01:50:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd7c101930 0x7efd7c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7b7fe640 1 -- 172.21.15.92:0/3929258851 learned_addr learned my addr 172.21.15.92:0/3929258851 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7affd640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd7c100740 0x7efd7c1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7b7fe640 1 -- 172.21.15.92:0/3929258851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd7c100740 msgr2=0x7efd7c1969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7b7fe640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd7c100740 0x7efd7c1969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7b7fe640 1 -- 172.21.15.92:0/3929258851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd7c101930 msgr2=0x7efd7c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7b7fe640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd7c101930 0x7efd7c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7b7fe640 1 -- 172.21.15.92:0/3929258851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c19e600 con 0x7efd7c1022f0 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd7b7fe640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 0x7efd7c19df90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd74037040 tx=0x7efd74032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd63fff640 1 -- 172.21.15.92:0/3929258851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74041020 con 0x7efd7c1022f0 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c19e890 con 0x7efd7c1022f0 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd63fff640 1 -- 172.21.15.92:0/3929258851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd74034d90 con 0x7efd7c1022f0 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd63fff640 1 -- 172.21.15.92:0/3929258851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74039040 con 0x7efd7c1022f0 2024-03-19T01:50:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.101+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c19edd0 con 0x7efd7c1022f0 2024-03-19T01:50:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.103+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c107a60 con 0x7efd7c1022f0 2024-03-19T01:50:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.104+0000 7efd63fff640 1 -- 172.21.15.92:0/3929258851 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7efd74044020 con 0x7efd7c1022f0 2024-03-19T01:50:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.104+0000 7efd63fff640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd640a2800 0x7efd640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.104+0000 7efd7affd640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd640a2800 0x7efd640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.105+0000 7efd63fff640 1 -- 172.21.15.92:0/3929258851 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efd740f0480 con 0x7efd7c1022f0 2024-03-19T01:50:10.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.105+0000 7efd7affd640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd640a2800 0x7efd640a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd6c005fd0 tx=0x7efd6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.109+0000 7efd63fff640 1 -- 172.21.15.92:0/3929258851 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efd7404b020 con 0x7efd7c1022f0 2024-03-19T01:50:10.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:10.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: Saving service mds.all spec with placement smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-03-19T01:50:10.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:10.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:50:10.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:10.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:10.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:10.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:10.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:10.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:10.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:09 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:10.287 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.287+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd7c062420 con 0x7efd7c1022f0 2024-03-19T01:50:10.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.288+0000 7efd63fff640 1 -- 172.21.15.92:0/3929258851 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18493 (secure 0 0 0) 0x7efd740bd9c0 con 0x7efd7c1022f0 2024-03-19T01:50:10.289 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:10.289 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":81,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:50:00.507820+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd640a2800 msgr2=0x7efd640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd640a2800 0x7efd640a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd6c005fd0 tx=0x7efd6c00b410 comp rx=0 tx=0).stop 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 msgr2=0x7efd7c19df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 0x7efd7c19df90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd74037040 tx=0x7efd74032f20 comp rx=0 tx=0).stop 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 shutdown_connections 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd7c1022f0 0x7efd7c19df90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd7c101930 0x7efd7c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd640a2800 0x7efd640a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 --2- 172.21.15.92:0/3929258851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd7c100740 0x7efd7c1969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 >> 172.21.15.92:0/3929258851 conn(0x7efd7c0fbcb0 msgr2=0x7efd7c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:10.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 shutdown_connections 2024-03-19T01:50:10.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.291+0000 7efd814a1640 1 -- 172.21.15.92:0/3929258851 wait complete. 2024-03-19T01:50:10.301 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:10.301 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: Saving service mds.all spec with placement smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-03-19T01:50:10.301 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:10.301 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:50:10.301 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:10.301 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:10.301 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:10.301 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:10.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:10.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:10.302 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:09 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:10.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:46:07.695753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:50:10.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:50:10.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.408+0000 7f997f885640 1 -- 172.21.15.92:0/682480856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978100a00 msgr2=0x7f9978100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.408+0000 7f997f885640 1 --2- 172.21.15.92:0/682480856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978100a00 0x7f9978100e80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9968009900 tx=0x7f996802bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.408+0000 7f997f885640 1 -- 172.21.15.92:0/682480856 shutdown_connections 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.408+0000 7f997f885640 1 --2- 172.21.15.92:0/682480856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99781013c0 0x7f9978108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.408+0000 7f997f885640 1 --2- 172.21.15.92:0/682480856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978100a00 0x7f9978100e80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.408+0000 7f997f885640 1 --2- 172.21.15.92:0/682480856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9978101cd0 0x7f99781020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.408+0000 7f997f885640 1 -- 172.21.15.92:0/682480856 >> 172.21.15.92:0/682480856 conn(0x7f99780fb450 msgr2=0x7f99780fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.408+0000 7f997f885640 1 -- 172.21.15.92:0/682480856 shutdown_connections 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.408+0000 7f997f885640 1 -- 172.21.15.92:0/682480856 wait complete. 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997f885640 1 Processor -- start 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997f885640 1 -- start start 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997f885640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9978100a00 0x7f9978100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997f885640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99781013c0 0x7f997819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.413 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997f885640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978101cd0 0x7f997819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997f885640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f997810c3d0 con 0x7f9978101cd0 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997f885640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f997810c250 con 0x7f99781013c0 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997f885640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f997810c550 con 0x7f9978100a00 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997de22640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978101cd0 0x7f997819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997de22640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978101cd0 0x7f997819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51270/0 (socket says 172.21.15.92:51270) 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997de22640 1 -- 172.21.15.92:0/3818577121 learned_addr learned my addr 172.21.15.92:0/3818577121 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997ce20640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99781013c0 0x7f997819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997d621640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9978100a00 0x7f9978100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997de22640 1 -- 172.21.15.92:0/3818577121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9978100a00 msgr2=0x7f9978100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997de22640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9978100a00 0x7f9978100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997de22640 1 -- 172.21.15.92:0/3818577121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99781013c0 msgr2=0x7f997819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997de22640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99781013c0 0x7f997819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.414 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997de22640 1 -- 172.21.15.92:0/3818577121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997819f7c0 con 0x7f9978101cd0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997d621640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9978100a00 0x7f9978100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997de22640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978101cd0 0x7f997819eeb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f997000eb10 tx=0x7f997000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f996e7fc640 1 -- 172.21.15.92:0/3818577121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970004b20 con 0x7f9978101cd0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.409+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997819fa50 con 0x7f9978101cd0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.410+0000 7f996e7fc640 1 -- 172.21.15.92:0/3818577121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9970004cc0 con 0x7f9978101cd0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.410+0000 7f996e7fc640 1 -- 172.21.15.92:0/3818577121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970010760 con 0x7f9978101cd0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.410+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997819ff70 con 0x7f9978101cd0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.410+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9978107bc0 con 0x7f9978101cd0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.411+0000 7f996e7fc640 1 -- 172.21.15.92:0/3818577121 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f9970005020 con 0x7f9978101cd0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.412+0000 7f996e7fc640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f99640a2800 0x7f99640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.412+0000 7f997d621640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f99640a2800 0x7f99640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.412+0000 7f996e7fc640 1 -- 172.21.15.92:0/3818577121 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f99700ced50 con 0x7f9978101cd0 2024-03-19T01:50:10.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.412+0000 7f997d621640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f99640a2800 0x7f99640a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9978069bf0 tx=0x7f997400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:10.416 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.416+0000 7f996e7fc640 1 -- 172.21.15.92:0/3818577121 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f99700d2030 con 0x7f9978101cd0 2024-03-19T01:50:10.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.590+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f997802d3f0 con 0x7f9978101cd0 2024-03-19T01:50:10.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.590+0000 7f996e7fc640 1 -- 172.21.15.92:0/3818577121 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f997009c290 con 0x7f9978101cd0 2024-03-19T01:50:10.591 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-03-19T01:50:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f99640a2800 msgr2=0x7f99640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f99640a2800 0x7f99640a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9978069bf0 tx=0x7f997400a560 comp rx=0 tx=0).stop 2024-03-19T01:50:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978101cd0 msgr2=0x7f997819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978101cd0 0x7f997819eeb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f997000eb10 tx=0x7f997000c2d0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 shutdown_connections 2024-03-19T01:50:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978101cd0 0x7f997819eeb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99781013c0 0x7f997819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f99640a2800 0x7f99640a4cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 --2- 172.21.15.92:0/3818577121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9978100a00 0x7f9978100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.592+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 >> 172.21.15.92:0/3818577121 conn(0x7f99780fb450 msgr2=0x7f9978104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.593+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 shutdown_connections 2024-03-19T01:50:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.593+0000 7f997f885640 1 -- 172.21.15.92:0/3818577121 wait complete. 2024-03-19T01:50:10.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.704+0000 7f2997227640 1 -- 172.21.15.92:0/1699593478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990103400 msgr2=0x7f2990109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.704+0000 7f2997227640 1 --2- 172.21.15.92:0/1699593478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990103400 0x7f2990109c90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2980009900 tx=0x7f298002bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.704+0000 7f2997227640 1 -- 172.21.15.92:0/1699593478 shutdown_connections 2024-03-19T01:50:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.704+0000 7f2997227640 1 --2- 172.21.15.92:0/1699593478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990103400 0x7f2990109c90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.704+0000 7f2997227640 1 --2- 172.21.15.92:0/1699593478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2990102a40 0x7f2990102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 --2- 172.21.15.92:0/1699593478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2990101850 0x7f2990101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 -- 172.21.15.92:0/1699593478 >> 172.21.15.92:0/1699593478 conn(0x7f29900fcdc0 msgr2=0x7f29900ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 -- 172.21.15.92:0/1699593478 shutdown_connections 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 -- 172.21.15.92:0/1699593478 wait complete. 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 Processor -- start 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 -- start start 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990101850 0x7f2990197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2990102a40 0x7f29901980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2990103400 0x7f299019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f299010d550 con 0x7f2990101850 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f299010d3d0 con 0x7f2990103400 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2997227640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f299010d6d0 con 0x7f2990102a40 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2994fc3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990101850 0x7f2990197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2994fc3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990101850 0x7f2990197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51284/0 (socket says 172.21.15.92:51284) 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f2994fc3640 1 -- 172.21.15.92:0/3586572646 learned_addr learned my addr 172.21.15.92:0/3586572646 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.705+0000 7f298ffff640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2990102a40 0x7f29901980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f2994fc3640 1 -- 172.21.15.92:0/3586572646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2990102a40 msgr2=0x7f29901980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f2994fc3640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2990102a40 0x7f29901980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f2994fc3640 1 -- 172.21.15.92:0/3586572646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2990103400 msgr2=0x7f299019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f2994fc3640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2990103400 0x7f299019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f2994fc3640 1 -- 172.21.15.92:0/3586572646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299019f7b0 con 0x7f2990101850 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f2994fc3640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990101850 0x7f2990197b90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f298800eaa0 tx=0x7f298800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f298dffb640 1 -- 172.21.15.92:0/3586572646 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988016070 con 0x7f2990101850 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f298dffb640 1 -- 172.21.15.92:0/3586572646 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298800ce20 con 0x7f2990101850 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f298dffb640 1 -- 172.21.15.92:0/3586572646 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988011470 con 0x7f2990101850 2024-03-19T01:50:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299019faa0 con 0x7f2990101850 2024-03-19T01:50:10.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.706+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299019ffa0 con 0x7f2990101850 2024-03-19T01:50:10.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.707+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2990108b10 con 0x7f2990101850 2024-03-19T01:50:10.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.708+0000 7f298dffb640 1 -- 172.21.15.92:0/3586572646 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f2988019070 con 0x7f2990101850 2024-03-19T01:50:10.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.709+0000 7f298dffb640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29780a2800 0x7f29780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:10.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.709+0000 7f298ffff640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29780a2800 0x7f29780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:10.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.709+0000 7f298dffb640 1 -- 172.21.15.92:0/3586572646 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f29880ce740 con 0x7f2990101850 2024-03-19T01:50:10.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.709+0000 7f298ffff640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29780a2800 0x7f29780a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f29901990a0 tx=0x7f2984009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.713+0000 7f298dffb640 1 -- 172.21.15.92:0/3586572646 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f298809bc80 con 0x7f2990101850 2024-03-19T01:50:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.930+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f299002d1f0 con 0x7f2990101850 2024-03-19T01:50:10.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.930+0000 7f298dffb640 1 -- 172.21.15.92:0/3586572646 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f298809b580 con 0x7f2990101850 2024-03-19T01:50:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:10.931 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-19T01:50:10.931 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 1 2024-03-19T01:50:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29780a2800 msgr2=0x7f29780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29780a2800 0x7f29780a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f29901990a0 tx=0x7f2984009110 comp rx=0 tx=0).stop 2024-03-19T01:50:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990101850 msgr2=0x7f2990197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990101850 0x7f2990197b90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f298800eaa0 tx=0x7f298800eea0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 shutdown_connections 2024-03-19T01:50:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2990103400 0x7f299019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f29780a2800 0x7f29780a4cc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2990102a40 0x7f29901980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 --2- 172.21.15.92:0/3586572646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990101850 0x7f2990197b90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:10.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 >> 172.21.15.92:0/3586572646 conn(0x7f29900fcdc0 msgr2=0x7f2990105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:10.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 shutdown_connections 2024-03-19T01:50:10.934 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:10.933+0000 7f2997227640 1 -- 172.21.15.92:0/3586572646 wait complete. 2024-03-19T01:50:11.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:10 smithi159 ceph-mon[67110]: Deploying daemon mds.l on smithi159 2024-03-19T01:50:11.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:10 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3929258851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:11.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:10 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3818577121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:50:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:10 smithi092 ceph-mon[62574]: Deploying daemon mds.l on smithi159 2024-03-19T01:50:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:10 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3929258851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:11.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:10 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3818577121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:50:11.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:10 smithi144 ceph-mon[67199]: Deploying daemon mds.l on smithi159 2024-03-19T01:50:11.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:10 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3929258851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:11.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:10 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3818577121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:50:12.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:11 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3586572646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:12.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:11 smithi159 ceph-mon[67110]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:12.206 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 01:50:12 smithi159 systemd[1]: Starting Ceph mds.l for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:50:12.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:11 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3586572646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:12.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:11 smithi092 ceph-mon[62574]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:12.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:11 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3586572646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:12.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:11 smithi144 ceph-mon[67199]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:12.584 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 01:50:12 smithi159 podman[97788]: 2024-03-19 01:50:12.192286245 +0000 UTC m=+0.017482545 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:50:12.584 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 01:50:12 smithi159 podman[97788]: 2024-03-19 01:50:12.339803596 +0000 UTC m=+0.164999881 container create 9f5dc9a82c9b118bcb5f0a0cbe84228f0403ee7601ea58ff775965d30539f97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T01:50:12.584 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 01:50:12 smithi159 podman[97788]: 2024-03-19 01:50:12.584239871 +0000 UTC m=+0.409436148 container init 9f5dc9a82c9b118bcb5f0a0cbe84228f0403ee7601ea58ff775965d30539f97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-19T01:50:12.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:12.955 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 01:50:12 smithi159 podman[97788]: 2024-03-19 01:50:12.586986874 +0000 UTC m=+0.412183154 container start 9f5dc9a82c9b118bcb5f0a0cbe84228f0403ee7601ea58ff775965d30539f97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:50:12.955 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 01:50:12 smithi159 ceph-mds[97808]: -- 172.21.15.159:0/3389314283 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c2b17ba40 con 0x556c2b14ec00 2024-03-19T01:50:12.955 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 01:50:12 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l[97804]: starting mds.l at 2024-03-19T01:50:12.955 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 01:50:12 smithi159 bash[97788]: 9f5dc9a82c9b118bcb5f0a0cbe84228f0403ee7601ea58ff775965d30539f97c 2024-03-19T01:50:12.955 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 01:50:12 smithi159 systemd[1]: Started Ceph mds.l for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:13.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.058+0000 7f866806e640 1 -- 172.21.15.92:0/1893735573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 msgr2=0x7f8660101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:13.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.058+0000 7f866806e640 1 --2- 172.21.15.92:0/1893735573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 0x7f8660101db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8650009900 tx=0x7f865002bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:13.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 -- 172.21.15.92:0/1893735573 shutdown_connections 2024-03-19T01:50:13.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 --2- 172.21.15.92:0/1893735573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86601022f0 0x7f8660108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 --2- 172.21.15.92:0/1893735573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 0x7f8660101db0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 --2- 172.21.15.92:0/1893735573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8660100740 0x7f8660100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 -- 172.21.15.92:0/1893735573 >> 172.21.15.92:0/1893735573 conn(0x7f86600fbcb0 msgr2=0x7f86600fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 -- 172.21.15.92:0/1893735573 shutdown_connections 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 -- 172.21.15.92:0/1893735573 wait complete. 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 Processor -- start 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 -- start start 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8660100740 0x7f8660199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 0x7f866019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86601022f0 0x7f86601a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f866010c3c0 con 0x7f8660101930 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f866010c240 con 0x7f8660100740 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.059+0000 7f866806e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f866010c540 con 0x7f86601022f0 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665609640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 0x7f866019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:13.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665609640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 0x7f866019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51296/0 (socket says 172.21.15.92:51296) 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665609640 1 -- 172.21.15.92:0/548099965 learned_addr learned my addr 172.21.15.92:0/548099965 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665e0a640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8660100740 0x7f8660199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f866660b640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86601022f0 0x7f86601a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665609640 1 -- 172.21.15.92:0/548099965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86601022f0 msgr2=0x7f86601a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665609640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86601022f0 0x7f86601a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665609640 1 -- 172.21.15.92:0/548099965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8660100740 msgr2=0x7f8660199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665609640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8660100740 0x7f8660199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665609640 1 -- 172.21.15.92:0/548099965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86601a1980 con 0x7f8660101930 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8665609640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 0x7f866019a2a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8650009230 tx=0x7f8650005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8656ffd640 1 -- 172.21.15.92:0/548099965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650041070 con 0x7f8660101930 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8656ffd640 1 -- 172.21.15.92:0/548099965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86500054e0 con 0x7f8660101930 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86601a1c10 con 0x7f8660101930 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f8656ffd640 1 -- 172.21.15.92:0/548099965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650034440 con 0x7f8660101930 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.060+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86601a2150 con 0x7f8660101930 2024-03-19T01:50:13.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.061+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86601a2320 con 0x7f8660101930 2024-03-19T01:50:13.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.062+0000 7f8656ffd640 1 -- 172.21.15.92:0/548099965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8650044070 con 0x7f8660101930 2024-03-19T01:50:13.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.063+0000 7f8656ffd640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f864c0a2880 0x7f864c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:13.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.063+0000 7f8665e0a640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f864c0a2880 0x7f864c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:13.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.063+0000 7f8656ffd640 1 -- 172.21.15.92:0/548099965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f86500f05a0 con 0x7f8660101930 2024-03-19T01:50:13.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.063+0000 7f8665e0a640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f864c0a2880 0x7f864c0a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8660101790 tx=0x7f865c009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:13.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.067+0000 7f8656ffd640 1 -- 172.21.15.92:0/548099965 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f86500bd470 con 0x7f8660101930 2024-03-19T01:50:13.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.279+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86601a2610 con 0x7f8660101930 2024-03-19T01:50:13.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.279+0000 7f8656ffd640 1 -- 172.21.15.92:0/548099965 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f86500c1510 con 0x7f8660101930 2024-03-19T01:50:13.280 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:13.280 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-19T01:50:13.280 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 2 2024-03-19T01:50:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f864c0a2880 msgr2=0x7f864c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f864c0a2880 0x7f864c0a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8660101790 tx=0x7f865c009670 comp rx=0 tx=0).stop 2024-03-19T01:50:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 msgr2=0x7f866019a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 0x7f866019a2a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8650009230 tx=0x7f8650005dd0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 shutdown_connections 2024-03-19T01:50:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f864c0a2880 0x7f864c0a4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86601022f0 0x7f86601a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.282 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8660101930 0x7f866019a2a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 --2- 172.21.15.92:0/548099965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8660100740 0x7f8660199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 >> 172.21.15.92:0/548099965 conn(0x7f86600fbcb0 msgr2=0x7f86600fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.281+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 shutdown_connections 2024-03-19T01:50:13.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:13.282+0000 7f866806e640 1 -- 172.21.15.92:0/548099965 wait complete. 2024-03-19T01:50:13.711 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6832/3252974913,v1:172.21.15.159:6833/3252974913] up:boot 2024-03-19T01:50:13.712 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: fsmap 1 up:standby 2024-03-19T01:50:13.712 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:50:13.712 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:13.712 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:13.712 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:13.712 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:13.712 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:13.712 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:13.712 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:13 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/548099965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:13.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6832/3252974913,v1:172.21.15.159:6833/3252974913] up:boot 2024-03-19T01:50:13.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: fsmap 1 up:standby 2024-03-19T01:50:13.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:50:13.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:13.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:13.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:13.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:13.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:13.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:13.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:13 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/548099965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:14.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6832/3252974913,v1:172.21.15.159:6833/3252974913] up:boot 2024-03-19T01:50:14.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: fsmap 1 up:standby 2024-03-19T01:50:14.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T01:50:14.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:14.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:14.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:14.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:14.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:14.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:14.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:13 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/548099965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:14.833 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:14 smithi144 ceph-mon[67199]: Deploying daemon mds.k on smithi144 2024-03-19T01:50:14.834 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:14 smithi144 ceph-mon[67199]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:14 smithi159 ceph-mon[67110]: Deploying daemon mds.k on smithi144 2024-03-19T01:50:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:14 smithi159 ceph-mon[67110]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:14 smithi092 ceph-mon[62574]: Deploying daemon mds.k on smithi144 2024-03-19T01:50:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:14 smithi092 ceph-mon[62574]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:15.096 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 01:50:15 smithi144 systemd[1]: Starting Ceph mds.k for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:50:15.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.406+0000 7f485e577640 1 -- 172.21.15.92:0/2789532959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4858103560 msgr2=0x7f4858109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.406+0000 7f485e577640 1 --2- 172.21.15.92:0/2789532959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4858103560 0x7f4858109df0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4850009cd0 tx=0x7f485002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.406+0000 7f485e577640 1 -- 172.21.15.92:0/2789532959 shutdown_connections 2024-03-19T01:50:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.406+0000 7f485e577640 1 --2- 172.21.15.92:0/2789532959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4858103560 0x7f4858109df0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.406+0000 7f485e577640 1 --2- 172.21.15.92:0/2789532959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4858102ba0 0x7f4858103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.406+0000 7f485e577640 1 --2- 172.21.15.92:0/2789532959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48580ffb30 0x7f48580fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.406+0000 7f485e577640 1 -- 172.21.15.92:0/2789532959 >> 172.21.15.92:0/2789532959 conn(0x7f48580fb440 msgr2=0x7f48580fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.406+0000 7f485e577640 1 -- 172.21.15.92:0/2789532959 shutdown_connections 2024-03-19T01:50:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.406+0000 7f485e577640 1 -- 172.21.15.92:0/2789532959 wait complete. 2024-03-19T01:50:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f485e577640 1 Processor -- start 2024-03-19T01:50:15.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f485e577640 1 -- start start 2024-03-19T01:50:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f485e577640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48580ffb30 0x7f4858197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f485e577640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4858102ba0 0x7f4858197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f485e577640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4858103560 0x7f485819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f485e577640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f485810d4d0 con 0x7f48580ffb30 2024-03-19T01:50:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f485e577640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f485810d350 con 0x7f4858103560 2024-03-19T01:50:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f485e577640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f485810d650 con 0x7f4858102ba0 2024-03-19T01:50:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f4857fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48580ffb30 0x7f4858197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f4857fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48580ffb30 0x7f4858197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52886/0 (socket says 172.21.15.92:52886) 2024-03-19T01:50:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f4857fff640 1 -- 172.21.15.92:0/3704897804 learned_addr learned my addr 172.21.15.92:0/3704897804 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f485cb14640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4858103560 0x7f485819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f48577fe640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4858102ba0 0x7f4858197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f4857fff640 1 -- 172.21.15.92:0/3704897804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4858102ba0 msgr2=0x7f4858197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.407+0000 7f4857fff640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4858102ba0 0x7f4858197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f4857fff640 1 -- 172.21.15.92:0/3704897804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4858103560 msgr2=0x7f485819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f4857fff640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4858103560 0x7f485819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f4857fff640 1 -- 172.21.15.92:0/3704897804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485819f670 con 0x7f48580ffb30 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f485cb14640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4858103560 0x7f485819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f4857fff640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48580ffb30 0x7f4858197a50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f484c00d950 tx=0x7f484c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f48557fa640 1 -- 172.21.15.92:0/3704897804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c017070 con 0x7f48580ffb30 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f48557fa640 1 -- 172.21.15.92:0/3704897804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f484c009c30 con 0x7f48580ffb30 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485819f960 con 0x7f48580ffb30 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f48557fa640 1 -- 172.21.15.92:0/3704897804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c00f600 con 0x7f48580ffb30 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.408+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485819fe60 con 0x7f48580ffb30 2024-03-19T01:50:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.409+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4858108c00 con 0x7f48580ffb30 2024-03-19T01:50:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.410+0000 7f48557fa640 1 -- 172.21.15.92:0/3704897804 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f484c01a070 con 0x7f48580ffb30 2024-03-19T01:50:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.410+0000 7f48557fa640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48400a2800 0x7f48400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.411+0000 7f48577fe640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48400a2800 0x7f48400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.411+0000 7f48557fa640 1 -- 172.21.15.92:0/3704897804 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f484c0ce020 con 0x7f48580ffb30 2024-03-19T01:50:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.411+0000 7f48577fe640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48400a2800 0x7f48400a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4848005e10 tx=0x7f4848005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:15.415 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.415+0000 7f48557fa640 1 -- 172.21.15.92:0/3704897804 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f484c009de0 con 0x7f48580ffb30 2024-03-19T01:50:15.498 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 01:50:15 smithi144 podman[96927]: 2024-03-19 01:50:15.132706022 +0000 UTC m=+0.017582887 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:50:15.498 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 01:50:15 smithi144 podman[96927]: 2024-03-19 01:50:15.236816025 +0000 UTC m=+0.121692880 container create e735324ea198e9bbc53f2178329f7ddc068ae07bab04d458ebf050de783842e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-19T01:50:15.498 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 01:50:15 smithi144 podman[96927]: 2024-03-19 01:50:15.472732446 +0000 UTC m=+0.357609311 container init e735324ea198e9bbc53f2178329f7ddc068ae07bab04d458ebf050de783842e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-19T01:50:15.498 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 01:50:15 smithi144 podman[96927]: 2024-03-19 01:50:15.47548014 +0000 UTC m=+0.360357009 container start e735324ea198e9bbc53f2178329f7ddc068ae07bab04d458ebf050de783842e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-19T01:50:15.628 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.628+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f485802d1f0 con 0x7f48580ffb30 2024-03-19T01:50:15.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.630+0000 7f48557fa640 1 -- 172.21.15.92:0/3704897804 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f484c09b560 con 0x7f48580ffb30 2024-03-19T01:50:15.630 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:15.630 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-19T01:50:15.631 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 2 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48400a2800 msgr2=0x7f48400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48400a2800 0x7f48400a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4848005e10 tx=0x7f4848005da0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48580ffb30 msgr2=0x7f4858197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48580ffb30 0x7f4858197a50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f484c00d950 tx=0x7f484c00de10 comp rx=0 tx=0).stop 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 shutdown_connections 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4858103560 0x7f485819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48400a2800 0x7f48400a4cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4858102ba0 0x7f4858197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 --2- 172.21.15.92:0/3704897804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48580ffb30 0x7f4858197a50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:15.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 >> 172.21.15.92:0/3704897804 conn(0x7f48580fb440 msgr2=0x7f4858105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:15.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 shutdown_connections 2024-03-19T01:50:15.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:15.632+0000 7f485e577640 1 -- 172.21.15.92:0/3704897804 wait complete. 2024-03-19T01:50:15.846 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 01:50:15 smithi144 ceph-mds[96946]: -- 172.21.15.144:0/3613082968 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641d1bcda40 con 0x5641d1ba1000 2024-03-19T01:50:15.846 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 01:50:15 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k[96942]: starting mds.k at 2024-03-19T01:50:15.846 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 01:50:15 smithi144 bash[96927]: e735324ea198e9bbc53f2178329f7ddc068ae07bab04d458ebf050de783842e9 2024-03-19T01:50:15.847 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 01:50:15 smithi144 systemd[1]: Started Ceph mds.k for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:16.904 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:16.904 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:16.904 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3704897804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:16.904 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:16.904 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.144:6832/3429474924,v1:172.21.15.144:6833/3429474924] up:boot 2024-03-19T01:50:16.904 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: fsmap 2 up:standby 2024-03-19T01:50:16.904 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:50:16.904 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:16.904 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:16.905 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:16.905 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3704897804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.144:6832/3429474924,v1:172.21.15.144:6833/3429474924] up:boot 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: fsmap 2 up:standby 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3704897804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:17.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:17.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.144:6832/3429474924,v1:172.21.15.144:6833/3429474924] up:boot 2024-03-19T01:50:17.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: fsmap 2 up:standby 2024-03-19T01:50:17.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T01:50:17.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:17.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:17.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:17.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:17.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:17.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.866+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/1972385038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f48104c90 msgr2=0x7f6f48107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:17.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.866+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/1972385038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f48104c90 0x7f6f48107080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009900 tx=0x7f6f3c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:17.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.866+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/1972385038 shutdown_connections 2024-03-19T01:50:17.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.866+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/1972385038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f481075c0 0x7f6f481099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:17.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.866+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/1972385038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f48104c90 0x7f6f48107080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.866+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/1972385038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f48102360 0x7f6f48104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.866+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/1972385038 >> 172.21.15.92:0/1972385038 conn(0x7f6f480fbcb0 msgr2=0x7f6f480fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/1972385038 shutdown_connections 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/1972385038 wait complete. 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 Processor -- start 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 -- start start 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f48102360 0x7f6f48197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f48104c90 0x7f6f481980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f481075c0 0x7f6f4819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810d4b0 con 0x7f6f481075c0 2024-03-19T01:50:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810d330 con 0x7f6f48102360 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.867+0000 7f6f4d8dc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810d630 con 0x7f6f48104c90 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f477fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f481075c0 0x7f6f4819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f477fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f481075c0 0x7f6f4819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52912/0 (socket says 172.21.15.92:52912) 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f477fe640 1 -- 172.21.15.92:0/3906260544 learned_addr learned my addr 172.21.15.92:0/3906260544 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f46ffd640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f48102360 0x7f6f48197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f467fc640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f48104c90 0x7f6f481980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f477fe640 1 -- 172.21.15.92:0/3906260544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f48104c90 msgr2=0x7f6f481980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f477fe640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f48104c90 0x7f6f481980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f477fe640 1 -- 172.21.15.92:0/3906260544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f48102360 msgr2=0x7f6f48197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f477fe640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f48102360 0x7f6f48197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f477fe640 1 -- 172.21.15.92:0/3906260544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4819f7d0 con 0x7f6f481075c0 2024-03-19T01:50:17.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f477fe640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f481075c0 0x7f6f4819f160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000eaa0 tx=0x7f6f4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:17.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f4c8da640 1 -- 172.21.15.92:0/3906260544 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000cdb0 con 0x7f6f481075c0 2024-03-19T01:50:17.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4819fac0 con 0x7f6f481075c0 2024-03-19T01:50:17.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f4c8da640 1 -- 172.21.15.92:0/3906260544 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f40005020 con 0x7f6f481075c0 2024-03-19T01:50:17.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f4c8da640 1 -- 172.21.15.92:0/3906260544 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400116b0 con 0x7f6f481075c0 2024-03-19T01:50:17.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.868+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4819ffe0 con 0x7f6f481075c0 2024-03-19T01:50:17.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.869+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f480fd7a0 con 0x7f6f481075c0 2024-03-19T01:50:17.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.870+0000 7f6f4c8da640 1 -- 172.21.15.92:0/3906260544 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f6f40005560 con 0x7f6f481075c0 2024-03-19T01:50:17.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.871+0000 7f6f4c8da640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6f300a2800 0x7f6f300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:17.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.871+0000 7f6f4c8da640 1 -- 172.21.15.92:0/3906260544 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6f400ceb30 con 0x7f6f481075c0 2024-03-19T01:50:17.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.871+0000 7f6f46ffd640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6f300a2800 0x7f6f300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:17.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.872+0000 7f6f46ffd640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6f300a2800 0x7f6f300a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6f38005fd0 tx=0x7f6f3800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:17.875 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:17.875+0000 7f6f4c8da640 1 -- 172.21.15.92:0/3906260544 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6f40016030 con 0x7f6f481075c0 2024-03-19T01:50:17.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:17 smithi159 ceph-mon[67110]: Deploying daemon mds.j on smithi092 2024-03-19T01:50:18.024 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:17 smithi092 ceph-mon[62574]: Deploying daemon mds.j on smithi092 2024-03-19T01:50:18.025 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 01:50:17 smithi092 systemd[1]: Starting Ceph mds.j for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:50:18.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:17 smithi144 ceph-mon[67199]: Deploying daemon mds.j on smithi092 2024-03-19T01:50:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.097+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f4802d3a0 con 0x7f6f481075c0 2024-03-19T01:50:18.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.097+0000 7f6f4c8da640 1 -- 172.21.15.92:0/3906260544 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f6f4009ba20 con 0x7f6f481075c0 2024-03-19T01:50:18.098 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:18.098 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-19T01:50:18.098 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 3 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6f300a2800 msgr2=0x7f6f300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6f300a2800 0x7f6f300a4cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6f38005fd0 tx=0x7f6f3800b390 comp rx=0 tx=0).stop 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f481075c0 msgr2=0x7f6f4819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f481075c0 0x7f6f4819f160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000eaa0 tx=0x7f6f4000ef60 comp rx=0 tx=0).stop 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 shutdown_connections 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f481075c0 0x7f6f4819f160 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6f300a2800 0x7f6f300a4cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f48104c90 0x7f6f481980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 --2- 172.21.15.92:0/3906260544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f48102360 0x7f6f48197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 >> 172.21.15.92:0/3906260544 conn(0x7f6f480fbcb0 msgr2=0x7f6f48102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 shutdown_connections 2024-03-19T01:50:18.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:18.099+0000 7f6f4d8dc640 1 -- 172.21.15.92:0/3906260544 wait complete. 2024-03-19T01:50:18.276 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 01:50:18 smithi092 podman[103215]: 2024-03-19 01:50:17.92758417 +0000 UTC m=+0.017881873 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:50:18.276 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 01:50:18 smithi092 podman[103215]: 2024-03-19 01:50:18.045673553 +0000 UTC m=+0.135971258 container create 91ba3f745ff1b0b69bd493d6c06fef9ba370e4b9f0a9f93330a7542b39e6c5fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-j, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-19T01:50:18.276 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 01:50:18 smithi092 podman[103215]: 2024-03-19 01:50:18.273530052 +0000 UTC m=+0.363827764 container init 91ba3f745ff1b0b69bd493d6c06fef9ba370e4b9f0a9f93330a7542b39e6c5fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-j, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:50:18.276 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 01:50:18 smithi092 podman[103215]: 2024-03-19 01:50:18.276115526 +0000 UTC m=+0.366413241 container start 91ba3f745ff1b0b69bd493d6c06fef9ba370e4b9f0a9f93330a7542b39e6c5fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-19T01:50:18.560 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-mds[103236]: -- 172.21.15.92:0/3540564136 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5d19c1a40 con 0x55f5d1994800 2024-03-19T01:50:18.560 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-j[103232]: starting mds.j at 2024-03-19T01:50:18.560 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 01:50:18 smithi092 bash[103215]: 91ba3f745ff1b0b69bd493d6c06fef9ba370e4b9f0a9f93330a7542b39e6c5fa 2024-03-19T01:50:18.560 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 01:50:18 smithi092 systemd[1]: Started Ceph mds.j for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:18 smithi159 ceph-mon[67110]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:18 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3906260544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:19.052 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-mon[62574]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:19.052 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3906260544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:19.052 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:19.052 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:19.053 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:19.053 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:19.053 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:19.053 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:18 smithi144 ceph-mon[67199]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:18 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3906260544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:19 smithi159 ceph-mon[67110]: Deploying daemon mds.d on smithi092 2024-03-19T01:50:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:19 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.92:6834/383124452,v1:172.21.15.92:6835/383124452] up:boot 2024-03-19T01:50:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:19 smithi159 ceph-mon[67110]: fsmap 3 up:standby 2024-03-19T01:50:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T01:50:19.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:19 smithi159 ceph-mon[67110]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:19 smithi092 ceph-mon[62574]: Deploying daemon mds.d on smithi092 2024-03-19T01:50:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:19 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.92:6834/383124452,v1:172.21.15.92:6835/383124452] up:boot 2024-03-19T01:50:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:19 smithi092 ceph-mon[62574]: fsmap 3 up:standby 2024-03-19T01:50:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T01:50:20.055 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:19 smithi092 ceph-mon[62574]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:19 smithi144 ceph-mon[67199]: Deploying daemon mds.d on smithi092 2024-03-19T01:50:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:19 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.92:6834/383124452,v1:172.21.15.92:6835/383124452] up:boot 2024-03-19T01:50:20.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:19 smithi144 ceph-mon[67199]: fsmap 3 up:standby 2024-03-19T01:50:20.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-19T01:50:20.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:19 smithi144 ceph-mon[67199]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:20.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:20.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1891567651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 msgr2=0x7ff3d0108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:20.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1891567651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 0x7ff3d0108b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009ca0 tx=0x7ff3c802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1891567651 shutdown_connections 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1891567651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 0x7ff3d0108b80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1891567651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3d0101930 0x7ff3d0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1891567651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0100740 0x7ff3d0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1891567651 >> 172.21.15.92:0/1891567651 conn(0x7ff3d00fbcb0 msgr2=0x7ff3d00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1891567651 shutdown_connections 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1891567651 wait complete. 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 Processor -- start 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.228+0000 7ff3d56ea640 1 -- start start 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3d56ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0100740 0x7ff3d0196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3d56ea640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3d0101930 0x7ff3d0196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3d56ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 0x7ff3d019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3d56ea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010c3a0 con 0x7ff3d01022f0 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3d56ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010c220 con 0x7ff3d0100740 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3d56ea640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010c520 con 0x7ff3d0101930 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 0x7ff3d019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 0x7ff3d019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52954/0 (socket says 172.21.15.92:52954) 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3cf7fe640 1 -- 172.21.15.92:0/1833790482 learned_addr learned my addr 172.21.15.92:0/1833790482 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3ceffd640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0100740 0x7ff3d0196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3c7fff640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3d0101930 0x7ff3d0196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3cf7fe640 1 -- 172.21.15.92:0/1833790482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3d0101930 msgr2=0x7ff3d0196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3cf7fe640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3d0101930 0x7ff3d0196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3cf7fe640 1 -- 172.21.15.92:0/1833790482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0100740 msgr2=0x7ff3d0196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3cf7fe640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0100740 0x7ff3d0196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3cf7fe640 1 -- 172.21.15.92:0/1833790482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d0068ab0 con 0x7ff3d01022f0 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3cf7fe640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 0x7ff3d019df10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8037040 tx=0x7ff3c8032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.229+0000 7ff3ccff9640 1 -- 172.21.15.92:0/1833790482 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8039040 con 0x7ff3d01022f0 2024-03-19T01:50:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.230+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d0068d40 con 0x7ff3d01022f0 2024-03-19T01:50:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.230+0000 7ff3ccff9640 1 -- 172.21.15.92:0/1833790482 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c8044070 con 0x7ff3d01022f0 2024-03-19T01:50:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.230+0000 7ff3ccff9640 1 -- 172.21.15.92:0/1833790482 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8004260 con 0x7ff3d01022f0 2024-03-19T01:50:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.230+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d0069280 con 0x7ff3d01022f0 2024-03-19T01:50:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.231+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d0107a00 con 0x7ff3d01022f0 2024-03-19T01:50:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.232+0000 7ff3ccff9640 1 -- 172.21.15.92:0/1833790482 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ff3c8003970 con 0x7ff3d01022f0 2024-03-19T01:50:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.233+0000 7ff3ccff9640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3bc0a2800 0x7ff3bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.233+0000 7ff3ceffd640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3bc0a2800 0x7ff3bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.233+0000 7ff3ccff9640 1 -- 172.21.15.92:0/1833790482 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff3c80f0c30 con 0x7ff3d01022f0 2024-03-19T01:50:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.233+0000 7ff3ceffd640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3bc0a2800 0x7ff3bc0a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0101790 tx=0x7ff3c000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:20.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.238+0000 7ff3ccff9640 1 -- 172.21.15.92:0/1833790482 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff3c80be170 con 0x7ff3d01022f0 2024-03-19T01:50:20.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.474+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3d00694e0 con 0x7ff3d01022f0 2024-03-19T01:50:20.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.475+0000 7ff3ccff9640 1 -- 172.21.15.92:0/1833790482 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7ff3c80bda70 con 0x7ff3d01022f0 2024-03-19T01:50:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:20.476 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-19T01:50:20.476 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 4 2024-03-19T01:50:20.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3bc0a2800 msgr2=0x7ff3bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:20.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3bc0a2800 0x7ff3bc0a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0101790 tx=0x7ff3c000a2d0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 msgr2=0x7ff3d019df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:20.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 0x7ff3d019df10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8037040 tx=0x7ff3c8032f90 comp rx=0 tx=0).stop 2024-03-19T01:50:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 shutdown_connections 2024-03-19T01:50:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d01022f0 0x7ff3d019df10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3bc0a2800 0x7ff3bc0a4cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3d0101930 0x7ff3d0196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 --2- 172.21.15.92:0/1833790482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3d0100740 0x7ff3d0196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 >> 172.21.15.92:0/1833790482 conn(0x7ff3d00fbcb0 msgr2=0x7ff3d0104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 shutdown_connections 2024-03-19T01:50:20.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:20.477+0000 7ff3d56ea640 1 -- 172.21.15.92:0/1833790482 wait complete. 2024-03-19T01:50:20.798 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:20 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1833790482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:20 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1833790482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:20 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1833790482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:21.310 INFO:journalctl@ceph.mds.d.smithi092.stdout:Mar 19 01:50:21 smithi092 systemd[1]: Started Ceph mds.d for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:22.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:22.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:22.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:22.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:22.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:22.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:22.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: Deploying daemon mds.e on smithi144 2024-03-19T01:50:22.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:22.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.92:6836/1871566039,v1:172.21.15.92:6837/1871566039] up:boot 2024-03-19T01:50:22.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: fsmap 4 up:standby 2024-03-19T01:50:22.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:22 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:50:22.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:22.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:22.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:22.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:22.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:22.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:22.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:22.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: Deploying daemon mds.e on smithi144 2024-03-19T01:50:22.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:22.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.92:6836/1871566039,v1:172.21.15.92:6837/1871566039] up:boot 2024-03-19T01:50:22.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: fsmap 4 up:standby 2024-03-19T01:50:22.521 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:22 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:50:22.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: Deploying daemon mds.e on smithi144 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.92:6836/1871566039,v1:172.21.15.92:6837/1871566039] up:boot 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: fsmap 4 up:standby 2024-03-19T01:50:22.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:22 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-19T01:50:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.612+0000 7f413ba81640 1 -- 172.21.15.92:0/33653244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134103a60 msgr2=0x7f4134103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.612+0000 7f413ba81640 1 --2- 172.21.15.92:0/33653244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134103a60 0x7f4134103e80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4130009ca0 tx=0x7f413002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.612+0000 7f413ba81640 1 -- 172.21.15.92:0/33653244 shutdown_connections 2024-03-19T01:50:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.612+0000 7f413ba81640 1 --2- 172.21.15.92:0/33653244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4134105610 0x7f413410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.612+0000 7f413ba81640 1 --2- 172.21.15.92:0/33653244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4134104c50 0x7f41341050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.612+0000 7f413ba81640 1 --2- 172.21.15.92:0/33653244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134103a60 0x7f4134103e80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.612+0000 7f413ba81640 1 -- 172.21.15.92:0/33653244 >> 172.21.15.92:0/33653244 conn(0x7f41340ff010 msgr2=0x7f4134101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.612+0000 7f413ba81640 1 -- 172.21.15.92:0/33653244 shutdown_connections 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.612+0000 7f413ba81640 1 -- 172.21.15.92:0/33653244 wait complete. 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413ba81640 1 Processor -- start 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413ba81640 1 -- start start 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413ba81640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4134103a60 0x7f4134199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413ba81640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4134104c50 0x7f413419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413ba81640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134105610 0x7f41341a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413ba81640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f413410f6b0 con 0x7f4134105610 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413ba81640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f413410f530 con 0x7f4134103a60 2024-03-19T01:50:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413ba81640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f413410f830 con 0x7f4134104c50 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413a01e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134105610 0x7f41341a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413a01e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134105610 0x7f41341a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52990/0 (socket says 172.21.15.92:52990) 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.613+0000 7f413a01e640 1 -- 172.21.15.92:0/1593209824 learned_addr learned my addr 172.21.15.92:0/1593209824 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413981d640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4134103a60 0x7f4134199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413901c640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4134104c50 0x7f413419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413a01e640 1 -- 172.21.15.92:0/1593209824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4134104c50 msgr2=0x7f413419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413a01e640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4134104c50 0x7f413419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413a01e640 1 -- 172.21.15.92:0/1593209824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4134103a60 msgr2=0x7f4134199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413a01e640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4134103a60 0x7f4134199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413a01e640 1 -- 172.21.15.92:0/1593209824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41341a1970 con 0x7f4134105610 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413981d640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4134103a60 0x7f4134199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413901c640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4134104c50 0x7f413419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:50:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413a01e640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134105610 0x7f41341a1300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f412c00e970 tx=0x7f412c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f412affd640 1 -- 172.21.15.92:0/1593209824 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c00ccc0 con 0x7f4134105610 2024-03-19T01:50:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f412affd640 1 -- 172.21.15.92:0/1593209824 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c00ce60 con 0x7f4134105610 2024-03-19T01:50:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41341a1c60 con 0x7f4134105610 2024-03-19T01:50:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f412affd640 1 -- 172.21.15.92:0/1593209824 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c010490 con 0x7f4134105610 2024-03-19T01:50:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.614+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41341a21b0 con 0x7f4134105610 2024-03-19T01:50:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.615+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413410ad80 con 0x7f4134105610 2024-03-19T01:50:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.616+0000 7f412affd640 1 -- 172.21.15.92:0/1593209824 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f412c005020 con 0x7f4134105610 2024-03-19T01:50:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.617+0000 7f412affd640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41200a2800 0x7f41200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.617+0000 7f413981d640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41200a2800 0x7f41200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.617+0000 7f412affd640 1 -- 172.21.15.92:0/1593209824 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f412c0ceb50 con 0x7f4134105610 2024-03-19T01:50:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.618+0000 7f413981d640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41200a2800 0x7f41200a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f413002c700 tx=0x7f41300325b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.622+0000 7f412affd640 1 -- 172.21.15.92:0/1593209824 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f412c09c090 con 0x7f4134105610 2024-03-19T01:50:22.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.836+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f413402d1f0 con 0x7f4134105610 2024-03-19T01:50:22.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.837+0000 7f412affd640 1 -- 172.21.15.92:0/1593209824 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f412c09b990 con 0x7f4134105610 2024-03-19T01:50:22.837 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:22.837 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-19T01:50:22.838 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 5 2024-03-19T01:50:22.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41200a2800 msgr2=0x7f41200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:22.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41200a2800 0x7f41200a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f413002c700 tx=0x7f41300325b0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134105610 msgr2=0x7f41341a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:22.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134105610 0x7f41341a1300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f412c00e970 tx=0x7f412c00ed70 comp rx=0 tx=0).stop 2024-03-19T01:50:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 shutdown_connections 2024-03-19T01:50:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4134105610 0x7f41341a1300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41200a2800 0x7f41200a4cc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4134104c50 0x7f413419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 --2- 172.21.15.92:0/1593209824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4134103a60 0x7f4134199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 >> 172.21.15.92:0/1593209824 conn(0x7f41340ff010 msgr2=0x7f4134107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 shutdown_connections 2024-03-19T01:50:22.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:22.839+0000 7f413ba81640 1 -- 172.21.15.92:0/1593209824 wait complete. 2024-03-19T01:50:23.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:23 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1593209824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:23.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:23 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1593209824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:23.597 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 01:50:23 smithi144 systemd[1]: Starting Ceph mds.e for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:50:23.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:23 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1593209824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:23.961 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 01:50:23 smithi144 podman[97248]: 2024-03-19 01:50:23.579600266 +0000 UTC m=+0.017897376 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:50:23.961 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 01:50:23 smithi144 podman[97248]: 2024-03-19 01:50:23.700853629 +0000 UTC m=+0.139150728 container create 05838348eb27c8d3e2fadbaddca5ee25d8b2742fe56de6ae79ff73d4ec9c31c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2) 2024-03-19T01:50:23.962 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 01:50:23 smithi144 podman[97248]: 2024-03-19 01:50:23.962241473 +0000 UTC m=+0.400538585 container init 05838348eb27c8d3e2fadbaddca5ee25d8b2742fe56de6ae79ff73d4ec9c31c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:50:24.235 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 01:50:23 smithi144 podman[97248]: 2024-03-19 01:50:23.965010268 +0000 UTC m=+0.403307370 container start 05838348eb27c8d3e2fadbaddca5ee25d8b2742fe56de6ae79ff73d4ec9c31c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-19T01:50:24.235 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 01:50:23 smithi144 ceph-mds[97267]: -- 172.21.15.144:0/367434014 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559bc85ba40 con 0x5559bc82ec00 2024-03-19T01:50:24.235 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 01:50:23 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e[97263]: starting mds.e at 2024-03-19T01:50:24.235 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 01:50:24 smithi144 bash[97248]: 05838348eb27c8d3e2fadbaddca5ee25d8b2742fe56de6ae79ff73d4ec9c31c2 2024-03-19T01:50:24.235 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 01:50:24 smithi144 systemd[1]: Started Ceph mds.e for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:24.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:24 smithi092 ceph-mon[62574]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:24.633 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:24.633 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:24.633 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:24.633 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:24.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:24.634 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:24.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:24 smithi144 ceph-mon[67199]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:24.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:24.634 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:24.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:24.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:24.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:24.635 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:24.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:24 smithi159 ceph-mon[67110]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:24.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:24.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:24.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:24.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:24.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:24.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:24.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:24.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.956+0000 7fb157aa1640 1 -- 172.21.15.92:0/1237175636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1501000f0 msgr2=0x7fb150100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.956+0000 7fb157aa1640 1 --2- 172.21.15.92:0/1237175636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1501000f0 0x7fb150100510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009900 tx=0x7fb14c02bc80 comp rx=0 tx=0).stop 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.956+0000 7fb157aa1640 1 -- 172.21.15.92:0/1237175636 shutdown_connections 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.956+0000 7fb157aa1640 1 --2- 172.21.15.92:0/1237175636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1501013f0 0x7fb150101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.956+0000 7fb157aa1640 1 --2- 172.21.15.92:0/1237175636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb150100a50 0x7fb150100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.956+0000 7fb157aa1640 1 --2- 172.21.15.92:0/1237175636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1501000f0 0x7fb150100510 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.956+0000 7fb157aa1640 1 -- 172.21.15.92:0/1237175636 >> 172.21.15.92:0/1237175636 conn(0x7fb1500fbcb0 msgr2=0x7fb1500fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.956+0000 7fb157aa1640 1 -- 172.21.15.92:0/1237175636 shutdown_connections 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.956+0000 7fb157aa1640 1 -- 172.21.15.92:0/1237175636 wait complete. 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb157aa1640 1 Processor -- start 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb157aa1640 1 -- start start 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb157aa1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1501000f0 0x7fb150199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb157aa1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb150100a50 0x7fb15019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb157aa1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1501013f0 0x7fb1501a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb157aa1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb15010f690 con 0x7fb150100a50 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb157aa1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb15010f510 con 0x7fb1501013f0 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb157aa1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb15010f810 con 0x7fb1501000f0 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15503c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb150100a50 0x7fb15019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15503c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb150100a50 0x7fb15019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37354/0 (socket says 172.21.15.92:37354) 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15503c640 1 -- 172.21.15.92:0/2735032043 learned_addr learned my addr 172.21.15.92:0/2735032043 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15583d640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1501000f0 0x7fb150199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15603e640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1501013f0 0x7fb1501a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15503c640 1 -- 172.21.15.92:0/2735032043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1501000f0 msgr2=0x7fb150199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15503c640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1501000f0 0x7fb150199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15503c640 1 -- 172.21.15.92:0/2735032043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1501013f0 msgr2=0x7fb1501a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15503c640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1501013f0 0x7fb1501a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15503c640 1 -- 172.21.15.92:0/2735032043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1501a1930 con 0x7fb150100a50 2024-03-19T01:50:24.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15603e640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1501013f0 0x7fb1501a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:24.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.957+0000 7fb15503c640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb150100a50 0x7fb15019a260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c920 tx=0x7fb14000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:24.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.958+0000 7fb146ffd640 1 -- 172.21.15.92:0/2735032043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140007c70 con 0x7fb150100a50 2024-03-19T01:50:24.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.958+0000 7fb146ffd640 1 -- 172.21.15.92:0/2735032043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb140007e10 con 0x7fb150100a50 2024-03-19T01:50:24.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.958+0000 7fb146ffd640 1 -- 172.21.15.92:0/2735032043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140018890 con 0x7fb150100a50 2024-03-19T01:50:24.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.958+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1501a1c20 con 0x7fb150100a50 2024-03-19T01:50:24.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.958+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1501a2170 con 0x7fb150100a50 2024-03-19T01:50:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.959+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb15010af70 con 0x7fb150100a50 2024-03-19T01:50:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.960+0000 7fb146ffd640 1 -- 172.21.15.92:0/2735032043 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb140018a70 con 0x7fb150100a50 2024-03-19T01:50:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.960+0000 7fb146ffd640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb13c0a2800 0x7fb13c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.960+0000 7fb15583d640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb13c0a2800 0x7fb13c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.960+0000 7fb146ffd640 1 -- 172.21.15.92:0/2735032043 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb1400ce820 con 0x7fb150100a50 2024-03-19T01:50:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.961+0000 7fb15583d640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb13c0a2800 0x7fb13c0a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb14c02c7a0 tx=0x7fb14c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:24.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:24.965+0000 7fb146ffd640 1 -- 172.21.15.92:0/2735032043 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb140014030 con 0x7fb150100a50 2024-03-19T01:50:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.184+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb15002d3a0 con 0x7fb150100a50 2024-03-19T01:50:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.185+0000 7fb146ffd640 1 -- 172.21.15.92:0/2735032043 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fb14000de20 con 0x7fb150100a50 2024-03-19T01:50:25.185 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:25.185 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-19T01:50:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 6 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb13c0a2800 msgr2=0x7fb13c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb13c0a2800 0x7fb13c0a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb14c02c7a0 tx=0x7fb14c005fd0 comp rx=0 tx=0).stop 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb150100a50 msgr2=0x7fb15019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb150100a50 0x7fb15019a260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c920 tx=0x7fb14000cde0 comp rx=0 tx=0).stop 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 shutdown_connections 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1501013f0 0x7fb1501a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb150100a50 0x7fb15019a260 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb13c0a2800 0x7fb13c0a4cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 --2- 172.21.15.92:0/2735032043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1501000f0 0x7fb150199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 >> 172.21.15.92:0/2735032043 conn(0x7fb1500fbcb0 msgr2=0x7fb1500fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 shutdown_connections 2024-03-19T01:50:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:25.187+0000 7fb157aa1640 1 -- 172.21.15.92:0/2735032043 wait complete. 2024-03-19T01:50:25.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:25 smithi159 ceph-mon[67110]: Deploying daemon mds.f on smithi159 2024-03-19T01:50:25.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:25 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.144:6834/2148025721,v1:172.21.15.144:6835/2148025721] up:boot 2024-03-19T01:50:25.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:25 smithi159 ceph-mon[67110]: fsmap 5 up:standby 2024-03-19T01:50:25.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:25 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:50:25.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:25 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2735032043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:25.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:25 smithi092 ceph-mon[62574]: Deploying daemon mds.f on smithi159 2024-03-19T01:50:25.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:25 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.144:6834/2148025721,v1:172.21.15.144:6835/2148025721] up:boot 2024-03-19T01:50:25.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:25 smithi092 ceph-mon[62574]: fsmap 5 up:standby 2024-03-19T01:50:25.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:25 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:50:25.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:25 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2735032043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:25.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:25 smithi144 ceph-mon[67199]: Deploying daemon mds.f on smithi159 2024-03-19T01:50:25.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:25 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.144:6834/2148025721,v1:172.21.15.144:6835/2148025721] up:boot 2024-03-19T01:50:25.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:25 smithi144 ceph-mon[67199]: fsmap 5 up:standby 2024-03-19T01:50:25.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:25 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T01:50:25.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:25 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2735032043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:26.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:26 smithi159 ceph-mon[67110]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:26.456 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 01:50:26 smithi159 systemd[1]: Starting Ceph mds.f for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:50:26.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:26 smithi092 ceph-mon[62574]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:26.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:26 smithi144 ceph-mon[67199]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:26.830 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 01:50:26 smithi159 podman[98104]: 2024-03-19 01:50:26.387133005 +0000 UTC m=+0.017917306 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:50:26.830 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 01:50:26 smithi159 podman[98104]: 2024-03-19 01:50:26.536630126 +0000 UTC m=+0.167414404 container create dc3a13d7a5c014ffbc06fc7623b5b7c7d4a6607ddf30a411df34d86fdbb3e162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:50:26.830 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 01:50:26 smithi159 podman[98104]: 2024-03-19 01:50:26.831232862 +0000 UTC m=+0.462017142 container init dc3a13d7a5c014ffbc06fc7623b5b7c7d4a6607ddf30a411df34d86fdbb3e162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:50:27.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:27.205 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 01:50:26 smithi159 podman[98104]: 2024-03-19 01:50:26.83391751 +0000 UTC m=+0.464701797 container start dc3a13d7a5c014ffbc06fc7623b5b7c7d4a6607ddf30a411df34d86fdbb3e162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-03-19T01:50:27.205 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 01:50:26 smithi159 ceph-mds[98123]: -- 172.21.15.159:0/2824806991 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bfbc937a40 con 0x55bfbc90ac00 2024-03-19T01:50:27.205 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 01:50:26 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f[98119]: starting mds.f at 2024-03-19T01:50:27.205 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 01:50:26 smithi159 bash[98104]: dc3a13d7a5c014ffbc06fc7623b5b7c7d4a6607ddf30a411df34d86fdbb3e162 2024-03-19T01:50:27.205 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 01:50:26 smithi159 systemd[1]: Started Ceph mds.f for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:27.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 -- 172.21.15.92:0/3314905558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e60100740 msgr2=0x7f7e60100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/3314905558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e60100740 0x7f7e60100b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009900 tx=0x7f7e5002bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 -- 172.21.15.92:0/3314905558 shutdown_connections 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/3314905558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e601022f0 0x7f7e60108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/3314905558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e60101930 0x7f7e60101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/3314905558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e60100740 0x7f7e60100b60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 -- 172.21.15.92:0/3314905558 >> 172.21.15.92:0/3314905558 conn(0x7f7e600fbcb0 msgr2=0x7f7e600fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 -- 172.21.15.92:0/3314905558 shutdown_connections 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 -- 172.21.15.92:0/3314905558 wait complete. 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 Processor -- start 2024-03-19T01:50:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.317+0000 7f7e66a0b640 1 -- start start 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e66a0b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e60100740 0x7f7e60196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e66a0b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e60101930 0x7f7e60196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e66a0b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e601022f0 0x7f7e6019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e66a0b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6010c390 con 0x7f7e601022f0 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e66a0b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6010c210 con 0x7f7e60100740 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e66a0b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6010c510 con 0x7f7e60101930 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e64fa8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e601022f0 0x7f7e6019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e64fa8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e601022f0 0x7f7e6019e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37374/0 (socket says 172.21.15.92:37374) 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e64fa8640 1 -- 172.21.15.92:0/4100532301 learned_addr learned my addr 172.21.15.92:0/4100532301 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e5f7fe640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e60101930 0x7f7e60196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e5ffff640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e60100740 0x7f7e60196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e64fa8640 1 -- 172.21.15.92:0/4100532301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e60101930 msgr2=0x7f7e60196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e64fa8640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e60101930 0x7f7e60196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e64fa8640 1 -- 172.21.15.92:0/4100532301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e60100740 msgr2=0x7f7e60196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e64fa8640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e60100740 0x7f7e60196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e64fa8640 1 -- 172.21.15.92:0/4100532301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6019e6b0 con 0x7f7e601022f0 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e5ffff640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e60100740 0x7f7e60196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e64fa8640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e601022f0 0x7f7e6019e040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800cad0 tx=0x7f7e5800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e5d7fa640 1 -- 172.21.15.92:0/4100532301 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800ae20 con 0x7f7e601022f0 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6019e9a0 con 0x7f7e601022f0 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e5d7fa640 1 -- 172.21.15.92:0/4100532301 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e58005020 con 0x7f7e601022f0 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.318+0000 7f7e5d7fa640 1 -- 172.21.15.92:0/4100532301 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800f6f0 con 0x7f7e601022f0 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.319+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6019eea0 con 0x7f7e601022f0 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.320+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e60107ae0 con 0x7f7e601022f0 2024-03-19T01:50:27.322 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.321+0000 7f7e5d7fa640 1 -- 172.21.15.92:0/4100532301 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7e580051d0 con 0x7f7e601022f0 2024-03-19T01:50:27.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.321+0000 7f7e5d7fa640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e480a2800 0x7f7e480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:27.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.321+0000 7f7e5ffff640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e480a2800 0x7f7e480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:27.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.321+0000 7f7e5d7fa640 1 -- 172.21.15.92:0/4100532301 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7e580ce5c0 con 0x7f7e601022f0 2024-03-19T01:50:27.323 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.322+0000 7f7e5ffff640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e480a2800 0x7f7e480a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7e5002c7b0 tx=0x7f7e50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:27.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.326+0000 7f7e5d7fa640 1 -- 172.21.15.92:0/4100532301 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7e58014030 con 0x7f7e601022f0 2024-03-19T01:50:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.537+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e6002d440 con 0x7f7e601022f0 2024-03-19T01:50:27.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.538+0000 7f7e5d7fa640 1 -- 172.21.15.92:0/4100532301 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f7e5809bb00 con 0x7f7e601022f0 2024-03-19T01:50:27.539 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:27.539 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-19T01:50:27.539 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e480a2800 msgr2=0x7f7e480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e480a2800 0x7f7e480a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7e5002c7b0 tx=0x7f7e50005fd0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e601022f0 msgr2=0x7f7e6019e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e601022f0 0x7f7e6019e040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800cad0 tx=0x7f7e5800cf90 comp rx=0 tx=0).stop 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 shutdown_connections 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e601022f0 0x7f7e6019e040 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7e480a2800 0x7f7e480a4cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e60101930 0x7f7e60196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 --2- 172.21.15.92:0/4100532301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e60100740 0x7f7e60196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:27.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 >> 172.21.15.92:0/4100532301 conn(0x7f7e600fbcb0 msgr2=0x7f7e60104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:27.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 shutdown_connections 2024-03-19T01:50:27.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:27.540+0000 7f7e66a0b640 1 -- 172.21.15.92:0/4100532301 wait complete. 2024-03-19T01:50:28.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:28.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:28.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:28.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:28.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:28.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:28.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: Deploying daemon mds.c on smithi159 2024-03-19T01:50:28.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] up:boot 2024-03-19T01:50:28.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: fsmap 6 up:standby 2024-03-19T01:50:28.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:50:28.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:28.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:27 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4100532301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:28.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: Deploying daemon mds.c on smithi159 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] up:boot 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: fsmap 6 up:standby 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:28.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:27 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4100532301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:28.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:28.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:28.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:28.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:28.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:28.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:28.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: Deploying daemon mds.c on smithi159 2024-03-19T01:50:28.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] up:boot 2024-03-19T01:50:28.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: fsmap 6 up:standby 2024-03-19T01:50:28.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T01:50:28.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:28.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:27 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4100532301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:29.425 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 01:50:29 smithi159 systemd[1]: Starting Ceph mds.c for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:50:29.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:29.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.668+0000 7f53f9a63640 1 -- 172.21.15.92:0/3010460528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f41075c0 msgr2=0x7f53f41099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:29.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.668+0000 7f53f9a63640 1 --2- 172.21.15.92:0/3010460528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f41075c0 0x7f53f41099b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009ca0 tx=0x7f53ec02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.668+0000 7f53f9a63640 1 -- 172.21.15.92:0/3010460528 shutdown_connections 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.668+0000 7f53f9a63640 1 --2- 172.21.15.92:0/3010460528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f41075c0 0x7f53f41099b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.668+0000 7f53f9a63640 1 --2- 172.21.15.92:0/3010460528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f4104c90 0x7f53f4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.668+0000 7f53f9a63640 1 --2- 172.21.15.92:0/3010460528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f4102360 0x7f53f4104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.668+0000 7f53f9a63640 1 -- 172.21.15.92:0/3010460528 >> 172.21.15.92:0/3010460528 conn(0x7f53f40fbcd0 msgr2=0x7f53f40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.668+0000 7f53f9a63640 1 -- 172.21.15.92:0/3010460528 shutdown_connections 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.668+0000 7f53f9a63640 1 -- 172.21.15.92:0/3010460528 wait complete. 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f9a63640 1 Processor -- start 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f9a63640 1 -- start start 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f9a63640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f4102360 0x7f53f4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f9a63640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f4104c90 0x7f53f4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f9a63640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f41075c0 0x7f53f419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f9a63640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410d4e0 con 0x7f53f4102360 2024-03-19T01:50:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f9a63640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410d360 con 0x7f53f41075c0 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f9a63640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53f410d660 con 0x7f53f4104c90 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f4102360 0x7f53f4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f4102360 0x7f53f4197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37394/0 (socket says 172.21.15.92:37394) 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53f37fe640 1 -- 172.21.15.92:0/1366648590 learned_addr learned my addr 172.21.15.92:0/1366648590 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.669+0000 7f53ebfff640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f4104c90 0x7f53f4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f3fff640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f41075c0 0x7f53f419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f37fe640 1 -- 172.21.15.92:0/1366648590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f4104c90 msgr2=0x7f53f4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f37fe640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f4104c90 0x7f53f4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f37fe640 1 -- 172.21.15.92:0/1366648590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f41075c0 msgr2=0x7f53f419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f37fe640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f41075c0 0x7f53f419f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f37fe640 1 -- 172.21.15.92:0/1366648590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f419f6c0 con 0x7f53f4102360 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f37fe640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f4102360 0x7f53f4197aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f53e400d950 tx=0x7f53e400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f17fa640 1 -- 172.21.15.92:0/1366648590 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4017030 con 0x7f53f4102360 2024-03-19T01:50:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f17fa640 1 -- 172.21.15.92:0/1366648590 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e4004020 con 0x7f53f4102360 2024-03-19T01:50:29.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f419f9b0 con 0x7f53f4102360 2024-03-19T01:50:29.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f17fa640 1 -- 172.21.15.92:0/1366648590 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4005650 con 0x7f53f4102360 2024-03-19T01:50:29.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.670+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f419ff00 con 0x7f53f4102360 2024-03-19T01:50:29.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.671+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f4068a60 con 0x7f53f4102360 2024-03-19T01:50:29.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.672+0000 7f53f17fa640 1 -- 172.21.15.92:0/1366648590 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f53e40041d0 con 0x7f53f4102360 2024-03-19T01:50:29.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.673+0000 7f53f17fa640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53e00a2800 0x7f53e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:29.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.673+0000 7f53ebfff640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53e00a2800 0x7f53e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:29.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.673+0000 7f53f17fa640 1 -- 172.21.15.92:0/1366648590 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f53e40ce070 con 0x7f53f4102360 2024-03-19T01:50:29.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.673+0000 7f53ebfff640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53e00a2800 0x7f53e00a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53f4198fb0 tx=0x7f53dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:29.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.677+0000 7f53f17fa640 1 -- 172.21.15.92:0/1366648590 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f53e400e350 con 0x7f53f4102360 2024-03-19T01:50:29.705 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 01:50:29 smithi159 podman[98403]: 2024-03-19 01:50:29.328001404 +0000 UTC m=+0.017737727 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:50:29.705 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 01:50:29 smithi159 podman[98403]: 2024-03-19 01:50:29.458738005 +0000 UTC m=+0.148474318 container create 75c7a3f9696f9855e08a807644356e4c8e7139cb9dca7d6f18823c0311ee690d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-19T01:50:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.892+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53f40fd7c0 con 0x7f53f4102360 2024-03-19T01:50:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.893+0000 7f53f17fa640 1 -- 172.21.15.92:0/1366648590 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f53f40fd7c0 con 0x7f53f4102360 2024-03-19T01:50:29.893 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:29.894 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-19T01:50:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2024-03-19T01:50:29.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53e00a2800 msgr2=0x7f53e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:29.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53e00a2800 0x7f53e00a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f53f4198fb0 tx=0x7f53dc00a2d0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f4102360 msgr2=0x7f53f4197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f4102360 0x7f53f4197aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f53e400d950 tx=0x7f53e400dd50 comp rx=0 tx=0).stop 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 shutdown_connections 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f41075c0 0x7f53f419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53e00a2800 0x7f53e00a4cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f4104c90 0x7f53f4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 --2- 172.21.15.92:0/1366648590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53f4102360 0x7f53f4197aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 >> 172.21.15.92:0/1366648590 conn(0x7f53f40fbcd0 msgr2=0x7f53f4102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 shutdown_connections 2024-03-19T01:50:29.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:29.895+0000 7f53f9a63640 1 -- 172.21.15.92:0/1366648590 wait complete. 2024-03-19T01:50:30.205 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 01:50:29 smithi159 podman[98403]: 2024-03-19 01:50:29.736881606 +0000 UTC m=+0.426617919 container init 75c7a3f9696f9855e08a807644356e4c8e7139cb9dca7d6f18823c0311ee690d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-19T01:50:30.205 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 01:50:29 smithi159 podman[98403]: 2024-03-19 01:50:29.739405442 +0000 UTC m=+0.429141751 container start 75c7a3f9696f9855e08a807644356e4c8e7139cb9dca7d6f18823c0311ee690d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2) 2024-03-19T01:50:30.205 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 01:50:29 smithi159 ceph-mds[98422]: -- 172.21.15.159:0/4168559013 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7cda69a40 con 0x55c7cda3cc00 2024-03-19T01:50:30.206 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 01:50:29 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c[98418]: starting mds.c at 2024-03-19T01:50:30.206 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 01:50:29 smithi159 bash[98403]: 75c7a3f9696f9855e08a807644356e4c8e7139cb9dca7d6f18823c0311ee690d 2024-03-19T01:50:30.206 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 01:50:29 smithi159 systemd[1]: Started Ceph mds.c for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:30.798 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:30 smithi144 ceph-mon[67199]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:30.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:30.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:30.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:30 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1366648590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:30.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:30.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:30.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:30.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:30 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:30 smithi092 ceph-mon[62574]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:30 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1366648590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:30 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:30.950 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:30 smithi159 ceph-mon[67110]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:30.950 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:30.950 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:30.950 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:30 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1366648590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:30.950 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:30.950 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:30.950 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:30.951 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:30 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:31 smithi092 ceph-mon[62574]: Deploying daemon mds.h on smithi144 2024-03-19T01:50:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:31 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] up:boot 2024-03-19T01:50:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:31 smithi092 ceph-mon[62574]: fsmap 7 up:standby 2024-03-19T01:50:31.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:31 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:50:31.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:31 smithi144 ceph-mon[67199]: Deploying daemon mds.h on smithi144 2024-03-19T01:50:31.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:31 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] up:boot 2024-03-19T01:50:31.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:31 smithi144 ceph-mon[67199]: fsmap 7 up:standby 2024-03-19T01:50:31.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:31 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:50:31.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:31.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:31 smithi159 ceph-mon[67110]: Deploying daemon mds.h on smithi144 2024-03-19T01:50:31.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:31 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] up:boot 2024-03-19T01:50:31.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:31 smithi159 ceph-mon[67110]: fsmap 7 up:standby 2024-03-19T01:50:31.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:31 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T01:50:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.021+0000 7fbed5590640 1 -- 172.21.15.92:0/872209003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0101930 msgr2=0x7fbed0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.021+0000 7fbed5590640 1 --2- 172.21.15.92:0/872209003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0101930 0x7fbed0101db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009900 tx=0x7fbec402bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.022+0000 7fbed5590640 1 -- 172.21.15.92:0/872209003 shutdown_connections 2024-03-19T01:50:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.022+0000 7fbed5590640 1 --2- 172.21.15.92:0/872209003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbed01022f0 0x7fbed0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.022+0000 7fbed5590640 1 --2- 172.21.15.92:0/872209003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0101930 0x7fbed0101db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.022+0000 7fbed5590640 1 --2- 172.21.15.92:0/872209003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed0100740 0x7fbed0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.022+0000 7fbed5590640 1 -- 172.21.15.92:0/872209003 >> 172.21.15.92:0/872209003 conn(0x7fbed00fbcb0 msgr2=0x7fbed00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.022+0000 7fbed5590640 1 -- 172.21.15.92:0/872209003 shutdown_connections 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.022+0000 7fbed5590640 1 -- 172.21.15.92:0/872209003 wait complete. 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.022+0000 7fbed5590640 1 Processor -- start 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.022+0000 7fbed5590640 1 -- start start 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbed5590640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0100740 0x7fbed0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbed5590640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbed0101930 0x7fbed0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbed5590640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed01022f0 0x7fbed019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbed5590640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbed010c3a0 con 0x7fbed0100740 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbed5590640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbed010c220 con 0x7fbed0101930 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbed5590640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbed010c520 con 0x7fbed01022f0 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeceffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0100740 0x7fbed0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:32.025 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeceffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0100740 0x7fbed0197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37416/0 (socket says 172.21.15.92:37416) 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeceffd640 1 -- 172.21.15.92:0/2527954361 learned_addr learned my addr 172.21.15.92:0/2527954361 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbece7fc640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbed0101930 0x7fbed0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbecf7fe640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed01022f0 0x7fbed019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeceffd640 1 -- 172.21.15.92:0/2527954361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed01022f0 msgr2=0x7fbed019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeceffd640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed01022f0 0x7fbed019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeceffd640 1 -- 172.21.15.92:0/2527954361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbed0101930 msgr2=0x7fbed0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeceffd640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbed0101930 0x7fbed0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeceffd640 1 -- 172.21.15.92:0/2527954361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed019f7e0 con 0x7fbed0100740 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeceffd640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0100740 0x7fbed0197bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbec000ea80 tx=0x7fbec000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeb7fff640 1 -- 172.21.15.92:0/2527954361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec000ce40 con 0x7fbed0100740 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed019fad0 con 0x7fbed0100740 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeb7fff640 1 -- 172.21.15.92:0/2527954361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec0011950 con 0x7fbed0100740 2024-03-19T01:50:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbeb7fff640 1 -- 172.21.15.92:0/2527954361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec001ab80 con 0x7fbed0100740 2024-03-19T01:50:32.027 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.023+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a0040 con 0x7fbed0100740 2024-03-19T01:50:32.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.024+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed01a0210 con 0x7fbed0100740 2024-03-19T01:50:32.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.026+0000 7fbeb7fff640 1 -- 172.21.15.92:0/2527954361 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fbec0011b00 con 0x7fbed0100740 2024-03-19T01:50:32.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.026+0000 7fbeb7fff640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbeb80a2800 0x7fbeb80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:32.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.026+0000 7fbece7fc640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbeb80a2800 0x7fbeb80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:32.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.026+0000 7fbeb7fff640 1 -- 172.21.15.92:0/2527954361 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbec00cee30 con 0x7fbed0100740 2024-03-19T01:50:32.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.027+0000 7fbece7fc640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbeb80a2800 0x7fbeb80a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbec402c7b0 tx=0x7fbec4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:32.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.030+0000 7fbeb7fff640 1 -- 172.21.15.92:0/2527954361 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbec0016030 con 0x7fbed0100740 2024-03-19T01:50:32.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.244+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbed01a03e0 con 0x7fbed0100740 2024-03-19T01:50:32.244 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.244+0000 7fbeb7fff640 1 -- 172.21.15.92:0/2527954361 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7fbec009c370 con 0x7fbed0100740 2024-03-19T01:50:32.245 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:32.245 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-19T01:50:32.245 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbeb80a2800 msgr2=0x7fbeb80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbeb80a2800 0x7fbeb80a4cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbec402c7b0 tx=0x7fbec4005fd0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0100740 msgr2=0x7fbed0197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0100740 0x7fbed0197bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbec000ea80 tx=0x7fbec000ef40 comp rx=0 tx=0).stop 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 shutdown_connections 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbeb80a2800 0x7fbeb80a4cc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbed01022f0 0x7fbed019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbed0101930 0x7fbed0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 --2- 172.21.15.92:0/2527954361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed0100740 0x7fbed0197bc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 >> 172.21.15.92:0/2527954361 conn(0x7fbed00fbcb0 msgr2=0x7fbed00fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 shutdown_connections 2024-03-19T01:50:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:32.246+0000 7fbed5590640 1 -- 172.21.15.92:0/2527954361 wait complete. 2024-03-19T01:50:32.347 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 01:50:32 smithi144 systemd[1]: Starting Ceph mds.h for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:50:32.659 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:32 smithi144 ceph-mon[67199]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:32.659 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:32 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2527954361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:32.659 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 01:50:32 smithi144 podman[97556]: 2024-03-19 01:50:32.270710568 +0000 UTC m=+0.017709009 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:50:32.659 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 01:50:32 smithi144 podman[97556]: 2024-03-19 01:50:32.398908673 +0000 UTC m=+0.145907124 container create b768b4be8e7b67e47959ab19205f16e252a6e8e9390cd6e9046181371d34ed99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True) 2024-03-19T01:50:32.660 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 01:50:32 smithi144 podman[97556]: 2024-03-19 01:50:32.659977248 +0000 UTC m=+0.406975688 container init b768b4be8e7b67e47959ab19205f16e252a6e8e9390cd6e9046181371d34ed99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T01:50:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:32 smithi092 ceph-mon[62574]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:32 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2527954361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:32 smithi159 ceph-mon[67110]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:32 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2527954361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:33.096 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 01:50:32 smithi144 podman[97556]: 2024-03-19 01:50:32.662719789 +0000 UTC m=+0.409718214 container start b768b4be8e7b67e47959ab19205f16e252a6e8e9390cd6e9046181371d34ed99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:50:33.096 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 01:50:32 smithi144 ceph-mds[97574]: -- 172.21.15.144:0/4206186565 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f839e65a40 con 0x55f839e38800 2024-03-19T01:50:33.097 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 01:50:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h[97570]: starting mds.h at 2024-03-19T01:50:33.097 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 01:50:32 smithi144 bash[97556]: b768b4be8e7b67e47959ab19205f16e252a6e8e9390cd6e9046181371d34ed99 2024-03-19T01:50:33.097 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 01:50:32 smithi144 systemd[1]: Started Ceph mds.h for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:34.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: Deploying daemon mds.i on smithi159 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.144:6836/1367568659,v1:172.21.15.144:6837/1367568659] up:boot 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: fsmap 8 up:standby 2024-03-19T01:50:34.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:33 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:50:34.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:34.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:34.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: Deploying daemon mds.i on smithi159 2024-03-19T01:50:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.144:6836/1367568659,v1:172.21.15.144:6837/1367568659] up:boot 2024-03-19T01:50:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: fsmap 8 up:standby 2024-03-19T01:50:34.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:33 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:50:34.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:34.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:34.286 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:34.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:34.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:34.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:34.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:34.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: Deploying daemon mds.i on smithi159 2024-03-19T01:50:34.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:34.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.144:6836/1367568659,v1:172.21.15.144:6837/1367568659] up:boot 2024-03-19T01:50:34.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: fsmap 8 up:standby 2024-03-19T01:50:34.287 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:33 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T01:50:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.369+0000 7fb2e2af1640 1 -- 172.21.15.92:0/2145885369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc102300 msgr2=0x7fb2dc108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.369+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/2145885369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc102300 0x7fb2dc108b90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009900 tx=0x7fb2cc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.369+0000 7fb2e2af1640 1 -- 172.21.15.92:0/2145885369 shutdown_connections 2024-03-19T01:50:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.369+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/2145885369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc102300 0x7fb2dc108b90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.369+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/2145885369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2dc101940 0x7fb2dc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.369+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/2145885369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc100750 0x7fb2dc100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.369+0000 7fb2e2af1640 1 -- 172.21.15.92:0/2145885369 >> 172.21.15.92:0/2145885369 conn(0x7fb2dc0fbd00 msgr2=0x7fb2dc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.369+0000 7fb2e2af1640 1 -- 172.21.15.92:0/2145885369 shutdown_connections 2024-03-19T01:50:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.369+0000 7fb2e2af1640 1 -- 172.21.15.92:0/2145885369 wait complete. 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e2af1640 1 Processor -- start 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e2af1640 1 -- start start 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e2af1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc100750 0x7fb2dc197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e2af1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc101940 0x7fb2dc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e2af1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2dc102300 0x7fb2dc19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e2af1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc10d590 con 0x7fb2dc101940 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e2af1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc10d410 con 0x7fb2dc102300 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e2af1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc10d710 con 0x7fb2dc100750 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc101940 0x7fb2dc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2dbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc101940 0x7fb2dc198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41586/0 (socket says 172.21.15.92:41586) 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2dbfff640 1 -- 172.21.15.92:0/221665059 learned_addr learned my addr 172.21.15.92:0/221665059 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e108e640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2dc102300 0x7fb2dc19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e088d640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc100750 0x7fb2dc197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2dbfff640 1 -- 172.21.15.92:0/221665059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc100750 msgr2=0x7fb2dc197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2dbfff640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc100750 0x7fb2dc197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2dbfff640 1 -- 172.21.15.92:0/221665059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2dc102300 msgr2=0x7fb2dc19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2dbfff640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2dc102300 0x7fb2dc19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2dbfff640 1 -- 172.21.15.92:0/221665059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc19f700 con 0x7fb2dc101940 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2dbfff640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc101940 0x7fb2dc198020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c920 tx=0x7fb2d000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2d9ffb640 1 -- 172.21.15.92:0/221665059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0007c70 con 0x7fb2dc101940 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2d9ffb640 1 -- 172.21.15.92:0/221665059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d0007e10 con 0x7fb2dc101940 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2d9ffb640 1 -- 172.21.15.92:0/221665059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0018890 con 0x7fb2dc101940 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.370+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc19f9f0 con 0x7fb2dc101940 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.371+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc19ff40 con 0x7fb2dc101940 2024-03-19T01:50:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.372+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2dc1079a0 con 0x7fb2dc101940 2024-03-19T01:50:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.373+0000 7fb2d9ffb640 1 -- 172.21.15.92:0/221665059 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb2d0018a70 con 0x7fb2dc101940 2024-03-19T01:50:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.373+0000 7fb2d9ffb640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2c40a2800 0x7fb2c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.374+0000 7fb2e088d640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2c40a2800 0x7fb2c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.374+0000 7fb2d9ffb640 1 -- 172.21.15.92:0/221665059 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb2d00ce710 con 0x7fb2dc101940 2024-03-19T01:50:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.374+0000 7fb2e088d640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2c40a2800 0x7fb2c40a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc1017a0 tx=0x7fb2d40094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.378+0000 7fb2d9ffb640 1 -- 172.21.15.92:0/221665059 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb2d0014030 con 0x7fb2dc101940 2024-03-19T01:50:34.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.593+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2dc02d3a0 con 0x7fb2dc101940 2024-03-19T01:50:34.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.593+0000 7fb2d9ffb640 1 -- 172.21.15.92:0/221665059 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7fb2d009bc50 con 0x7fb2dc101940 2024-03-19T01:50:34.594 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:34.594 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-19T01:50:34.594 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 9 2024-03-19T01:50:34.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2c40a2800 msgr2=0x7fb2c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:34.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2c40a2800 0x7fb2c40a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc1017a0 tx=0x7fb2d40094f0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc101940 msgr2=0x7fb2dc198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:34.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc101940 0x7fb2dc198020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c920 tx=0x7fb2d000cde0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 shutdown_connections 2024-03-19T01:50:34.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2dc102300 0x7fb2dc19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc101940 0x7fb2dc198020 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2c40a2800 0x7fb2c40a4cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 --2- 172.21.15.92:0/221665059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc100750 0x7fb2dc197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:34.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 >> 172.21.15.92:0/221665059 conn(0x7fb2dc0fbd00 msgr2=0x7fb2dc104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:34.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 shutdown_connections 2024-03-19T01:50:34.597 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:34.595+0000 7fb2e2af1640 1 -- 172.21.15.92:0/221665059 wait complete. 2024-03-19T01:50:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:34 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/221665059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:35.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:34 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/221665059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:35.206 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:35 smithi159 systemd[1]: Starting Ceph mds.i for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T01:50:35.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:34 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/221665059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:35.545 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:35 smithi159 podman[98711]: 2024-03-19 01:50:35.161069248 +0000 UTC m=+0.018083204 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:50:35.545 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:35 smithi159 podman[98711]: 2024-03-19 01:50:35.276068639 +0000 UTC m=+0.133082623 container create 37fa8cc2c9bff735f7a2c71a7c4eca232fdb7ce37efa309b5b58d97d81b213bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-19T01:50:35.545 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:35 smithi159 podman[98711]: 2024-03-19 01:50:35.545920971 +0000 UTC m=+0.402934911 container init 37fa8cc2c9bff735f7a2c71a7c4eca232fdb7ce37efa309b5b58d97d81b213bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-19T01:50:35.856 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:35 smithi159 podman[98711]: 2024-03-19 01:50:35.548834436 +0000 UTC m=+0.405848371 container start 37fa8cc2c9bff735f7a2c71a7c4eca232fdb7ce37efa309b5b58d97d81b213bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307) 2024-03-19T01:50:35.856 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:35 smithi159 ceph-mds[98730]: -- 172.21.15.159:0/420843927 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a66fa5a40 con 0x559a66f78c00 2024-03-19T01:50:35.856 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: starting mds.i at 2024-03-19T01:50:35.856 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:35 smithi159 bash[98711]: 37fa8cc2c9bff735f7a2c71a7c4eca232fdb7ce37efa309b5b58d97d81b213bf 2024-03-19T01:50:35.856 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:35 smithi159 systemd[1]: Started Ceph mds.i for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:36.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:35 smithi092 ceph-mon[62574]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:36.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:35 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:36.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:35 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:36.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:35 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:36.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:35 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:36.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:35 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:36.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:35 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:36.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:35 smithi159 ceph-mon[67110]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:36.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:35 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:36.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:35 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:36.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:35 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:36.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:35 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:36.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:35 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:36.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:35 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:36.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:35 smithi144 ceph-mon[67199]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:36.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:35 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:36.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:35 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:36.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:35 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:36.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:35 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:36.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:35 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:36.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:35 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:36.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.714+0000 7f3136662640 1 -- 172.21.15.92:0/2769511845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010de00 msgr2=0x7f3130114690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.714+0000 7f3136662640 1 --2- 172.21.15.92:0/2769511845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010de00 0x7f3130114690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3128009ca0 tx=0x7f312802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.714+0000 7f3136662640 1 -- 172.21.15.92:0/2769511845 shutdown_connections 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.714+0000 7f3136662640 1 --2- 172.21.15.92:0/2769511845 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010de00 0x7f3130114690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.714+0000 7f3136662640 1 --2- 172.21.15.92:0/2769511845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f313010d440 0x7f313010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.714+0000 7f3136662640 1 --2- 172.21.15.92:0/2769511845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313010c250 0x7f313010c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.714+0000 7f3136662640 1 -- 172.21.15.92:0/2769511845 >> 172.21.15.92:0/2769511845 conn(0x7f313006c2c0 msgr2=0x7f313006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.714+0000 7f3136662640 1 -- 172.21.15.92:0/2769511845 shutdown_connections 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.714+0000 7f3136662640 1 -- 172.21.15.92:0/2769511845 wait complete. 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f3136662640 1 Processor -- start 2024-03-19T01:50:36.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f3136662640 1 -- start start 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f3136662640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313010c250 0x7f31301a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f3136662640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010d440 0x7f31301a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f3136662640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f313010de00 0x7f31301a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f3136662640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3130117ee0 con 0x7f313010d440 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f3136662640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3130117d60 con 0x7f313010c250 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f3136662640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3130118060 con 0x7f313010de00 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010d440 0x7f31301a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010d440 0x7f31301a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41614/0 (socket says 172.21.15.92:41614) 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312f7fe640 1 -- 172.21.15.92:0/3120240664 learned_addr learned my addr 172.21.15.92:0/3120240664 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312ffff640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313010c250 0x7f31301a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f3134bff640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f313010de00 0x7f31301a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312f7fe640 1 -- 172.21.15.92:0/3120240664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f313010de00 msgr2=0x7f31301a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312f7fe640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f313010de00 0x7f31301a9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312f7fe640 1 -- 172.21.15.92:0/3120240664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313010c250 msgr2=0x7f31301a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:36.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312f7fe640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313010c250 0x7f31301a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312f7fe640 1 -- 172.21.15.92:0/3120240664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31301aa130 con 0x7f313010d440 2024-03-19T01:50:36.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.715+0000 7f312f7fe640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010d440 0x7f31301a2a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f312400e9f0 tx=0x7f312400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:36.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.716+0000 7f312d7fa640 1 -- 172.21.15.92:0/3120240664 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312400cd80 con 0x7f313010d440 2024-03-19T01:50:36.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.716+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31301aa420 con 0x7f313010d440 2024-03-19T01:50:36.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.716+0000 7f312d7fa640 1 -- 172.21.15.92:0/3120240664 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124010910 con 0x7f313010d440 2024-03-19T01:50:36.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.716+0000 7f312d7fa640 1 -- 172.21.15.92:0/3120240664 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124018990 con 0x7f313010d440 2024-03-19T01:50:36.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.716+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31301aa9c0 con 0x7f313010d440 2024-03-19T01:50:36.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.717+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3130113570 con 0x7f313010d440 2024-03-19T01:50:36.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.718+0000 7f312d7fa640 1 -- 172.21.15.92:0/3120240664 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f3124010ac0 con 0x7f313010d440 2024-03-19T01:50:36.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.718+0000 7f312d7fa640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31180a2880 0x7f31180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:36.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.718+0000 7f312ffff640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31180a2880 0x7f31180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:36.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.719+0000 7f312d7fa640 1 -- 172.21.15.92:0/3120240664 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f31240cebb0 con 0x7f313010d440 2024-03-19T01:50:36.719 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.719+0000 7f312ffff640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31180a2880 0x7f31180a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f313010d2a0 tx=0x7f3120005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:36.722 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.723+0000 7f312d7fa640 1 -- 172.21.15.92:0/3120240664 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f312409c0f0 con 0x7f313010d440 2024-03-19T01:50:36.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.943+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f313002d1f0 con 0x7f313010d440 2024-03-19T01:50:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.944+0000 7f312d7fa640 1 -- 172.21.15.92:0/3120240664 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f312409b9f0 con 0x7f313010d440 2024-03-19T01:50:36.943 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:36.943 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-19T01:50:36.943 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 10 2024-03-19T01:50:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31180a2880 msgr2=0x7f31180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31180a2880 0x7f31180a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f313010d2a0 tx=0x7f3120005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010d440 msgr2=0x7f31301a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010d440 0x7f31301a2a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f312400e9f0 tx=0x7f312400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 shutdown_connections 2024-03-19T01:50:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31180a2880 0x7f31180a4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f313010de00 0x7f31301a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f313010d440 0x7f31301a2a50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 --2- 172.21.15.92:0/3120240664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313010c250 0x7f31301a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 >> 172.21.15.92:0/3120240664 conn(0x7f313006c2c0 msgr2=0x7f3130110740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 shutdown_connections 2024-03-19T01:50:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:36.946+0000 7f3136662640 1 -- 172.21.15.92:0/3120240664 wait complete. 2024-03-19T01:50:36.955 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:36 smithi092 ceph-mon[62574]: Deploying daemon mds.g on smithi092 2024-03-19T01:50:36.955 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:36 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] up:boot 2024-03-19T01:50:36.955 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:36 smithi092 ceph-mon[62574]: fsmap 9 up:standby 2024-03-19T01:50:36.956 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:36 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:50:37.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:36 smithi159 ceph-mon[67110]: Deploying daemon mds.g on smithi092 2024-03-19T01:50:37.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:36 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] up:boot 2024-03-19T01:50:37.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:36 smithi159 ceph-mon[67110]: fsmap 9 up:standby 2024-03-19T01:50:37.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:36 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:50:37.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:36 smithi144 ceph-mon[67199]: Deploying daemon mds.g on smithi092 2024-03-19T01:50:37.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:36 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] up:boot 2024-03-19T01:50:37.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:36 smithi144 ceph-mon[67199]: fsmap 9 up:standby 2024-03-19T01:50:37.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:36 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T01:50:38.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:37 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3120240664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:38.010 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:37 smithi092 ceph-mon[62574]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:37 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3120240664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:38.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:37 smithi159 ceph-mon[67110]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:38.278 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 01:50:38 smithi092 podman[104172]: 2024-03-19 01:50:38.255223258 +0000 UTC m=+0.360491534 container start 7136f2cd665d249609365725027a761d77af6cba97b9811afde51e76f90527fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-19T01:50:38.278 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 01:50:38 smithi092 ceph-mds[104191]: -- 172.21.15.92:0/2215656905 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e032f9a40 con 0x561e032ccc00 2024-03-19T01:50:38.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:37 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3120240664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:38.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:37 smithi144 ceph-mon[67199]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:38.560 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 01:50:38 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g[104187]: starting mds.g at 2024-03-19T01:50:38.560 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 01:50:38 smithi092 bash[104172]: 7136f2cd665d249609365725027a761d77af6cba97b9811afde51e76f90527fe 2024-03-19T01:50:38.560 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 01:50:38 smithi092 systemd[1]: Started Ceph mds.g for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:38.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 -- 172.21.15.92:0/1740254287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 msgr2=0x7f84b8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 --2- 172.21.15.92:0/1740254287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 0x7f84b8101db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009ca0 tx=0x7f84a002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 -- 172.21.15.92:0/1740254287 shutdown_connections 2024-03-19T01:50:39.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 --2- 172.21.15.92:0/1740254287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84b81022f0 0x7f84b8108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 --2- 172.21.15.92:0/1740254287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 0x7f84b8101db0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 --2- 172.21.15.92:0/1740254287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b8100740 0x7f84b8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 -- 172.21.15.92:0/1740254287 >> 172.21.15.92:0/1740254287 conn(0x7f84b80fbcb0 msgr2=0x7f84b80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 -- 172.21.15.92:0/1740254287 shutdown_connections 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 -- 172.21.15.92:0/1740254287 wait complete. 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.063+0000 7f84beeb5640 1 Processor -- start 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84beeb5640 1 -- start start 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84beeb5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84b8100740 0x7f84b80694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84beeb5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 0x7f84b819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84beeb5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b81022f0 0x7f84b819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84beeb5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810c3a0 con 0x7f84b8101930 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84beeb5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810c220 con 0x7f84b81022f0 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84beeb5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810c520 con 0x7f84b8100740 2024-03-19T01:50:39.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 0x7f84b819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 0x7f84b819c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41670/0 (socket says 172.21.15.92:41670) 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b7fff640 1 -- 172.21.15.92:0/2470258195 learned_addr learned my addr 172.21.15.92:0/2470258195 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84bcc51640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84b8100740 0x7f84b80694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84bd452640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b81022f0 0x7f84b819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b7fff640 1 -- 172.21.15.92:0/2470258195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84b8100740 msgr2=0x7f84b80694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b7fff640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84b8100740 0x7f84b80694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b7fff640 1 -- 172.21.15.92:0/2470258195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b81022f0 msgr2=0x7f84b819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b7fff640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b81022f0 0x7f84b819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b7fff640 1 -- 172.21.15.92:0/2470258195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b819f7f0 con 0x7f84b8101930 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b7fff640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 0x7f84b819c8c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009dc0 tx=0x7f84a00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84b5ffb640 1 -- 172.21.15.92:0/2470258195 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0039040 con 0x7f84b8101930 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.064+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b819fa20 con 0x7f84b8101930 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.065+0000 7f84b5ffb640 1 -- 172.21.15.92:0/2470258195 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a0044070 con 0x7f84b8101930 2024-03-19T01:50:39.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.065+0000 7f84b5ffb640 1 -- 172.21.15.92:0/2470258195 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0004310 con 0x7f84b8101930 2024-03-19T01:50:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.065+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b819ff60 con 0x7f84b8101930 2024-03-19T01:50:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.066+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b8107a60 con 0x7f84b8101930 2024-03-19T01:50:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.067+0000 7f84b5ffb640 1 -- 172.21.15.92:0/2470258195 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f84a0003a00 con 0x7f84b8101930 2024-03-19T01:50:39.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.067+0000 7f84b5ffb640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f84a40a2800 0x7f84a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.068+0000 7f84bcc51640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f84a40a2800 0x7f84a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.068+0000 7f84b5ffb640 1 -- 172.21.15.92:0/2470258195 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f84a00f1390 con 0x7f84b8101930 2024-03-19T01:50:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.068+0000 7f84bcc51640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f84a40a2800 0x7f84a40a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84b8101790 tx=0x7f84a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.071+0000 7f84b5ffb640 1 -- 172.21.15.92:0/2470258195 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f84a00be960 con 0x7f84b8101930 2024-03-19T01:50:39.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.291+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84b802d1f0 con 0x7f84b8101930 2024-03-19T01:50:39.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.291+0000 7f84b5ffb640 1 -- 172.21.15.92:0/2470258195 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f84a00be260 con 0x7f84b8101930 2024-03-19T01:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:39.292 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-19T01:50:39.292 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 11 2024-03-19T01:50:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f84a40a2800 msgr2=0x7f84a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f84a40a2800 0x7f84a40a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84b8101790 tx=0x7f84a8005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 msgr2=0x7f84b819c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 0x7f84b819c8c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009dc0 tx=0x7f84a00093f0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 shutdown_connections 2024-03-19T01:50:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b81022f0 0x7f84b819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b8101930 0x7f84b819c8c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f84a40a2800 0x7f84a40a4cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 --2- 172.21.15.92:0/2470258195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84b8100740 0x7f84b80694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 >> 172.21.15.92:0/2470258195 conn(0x7f84b80fbcb0 msgr2=0x7f84b8104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 shutdown_connections 2024-03-19T01:50:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:39.294+0000 7f84beeb5640 1 -- 172.21.15.92:0/2470258195 wait complete. 2024-03-19T01:50:39.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:39.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:39.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:39.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:39.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:39.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:39.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: Deploying daemon mds.a on smithi092 2024-03-19T01:50:39.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.92:6838/1427243227,v1:172.21.15.92:6839/1427243227] up:boot 2024-03-19T01:50:39.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: fsmap 10 up:standby 2024-03-19T01:50:39.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T01:50:39.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:39 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2470258195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:39.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: Deploying daemon mds.a on smithi092 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.92:6838/1427243227,v1:172.21.15.92:6839/1427243227] up:boot 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: fsmap 10 up:standby 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T01:50:39.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:39 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2470258195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: Deploying daemon mds.a on smithi092 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.92:6838/1427243227,v1:172.21.15.92:6839/1427243227] up:boot 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: fsmap 10 up:standby 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T01:50:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:39 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2470258195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:40.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:40 smithi159 ceph-mon[67110]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:40.733 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:40 smithi092 ceph-mon[62574]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:40.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:40 smithi144 ceph-mon[67199]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:41.060 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 01:50:40 smithi092 podman[104532]: 2024-03-19 01:50:40.97280425 +0000 UTC m=+0.137021207 container create 93b782d6d8b8373ea2e3df52471eef7dc082298bb4a39e5c7d4c64d25686ea64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-19T01:50:41.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.420+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/225631075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf681022f0 msgr2=0x7fcf68108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.420+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/225631075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf681022f0 0x7fcf68108b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009ca0 tx=0x7fcf6002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.420+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/225631075 shutdown_connections 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.420+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/225631075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf681022f0 0x7fcf68108b80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.420+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/225631075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf68101930 0x7fcf68101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.420+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/225631075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf68100740 0x7fcf68100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.420+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/225631075 >> 172.21.15.92:0/225631075 conn(0x7fcf680fbcb0 msgr2=0x7fcf680fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/225631075 shutdown_connections 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/225631075 wait complete. 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 Processor -- start 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 -- start start 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf68100740 0x7fcf68199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:41.423 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf68101930 0x7fcf6819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf681022f0 0x7fcf681a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810c3a0 con 0x7fcf68101930 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810c220 con 0x7fcf68100740 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6f9f4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810c520 con 0x7fcf681022f0 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6cf8f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf68101930 0x7fcf6819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6cf8f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf68101930 0x7fcf6819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41724/0 (socket says 172.21.15.92:41724) 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6cf8f640 1 -- 172.21.15.92:0/1844668653 learned_addr learned my addr 172.21.15.92:0/1844668653 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.421+0000 7fcf6d790640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf68100740 0x7fcf68199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6df91640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf681022f0 0x7fcf681a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6cf8f640 1 -- 172.21.15.92:0/1844668653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf681022f0 msgr2=0x7fcf681a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6cf8f640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf681022f0 0x7fcf681a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6cf8f640 1 -- 172.21.15.92:0/1844668653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf68100740 msgr2=0x7fcf68199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6cf8f640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf68100740 0x7fcf68199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6cf8f640 1 -- 172.21.15.92:0/1844668653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf681a1920 con 0x7fcf68101930 2024-03-19T01:50:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6d790640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf68100740 0x7fcf68199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:50:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6df91640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf681022f0 0x7fcf681a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6cf8f640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf68101930 0x7fcf6819a250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800d950 tx=0x7fcf5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf5e7fc640 1 -- 172.21.15.92:0/1844668653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58017030 con 0x7fcf68101930 2024-03-19T01:50:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf681a1c10 con 0x7fcf68101930 2024-03-19T01:50:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf5e7fc640 1 -- 172.21.15.92:0/1844668653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf58004020 con 0x7fcf68101930 2024-03-19T01:50:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf5e7fc640 1 -- 172.21.15.92:0/1844668653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf580047a0 con 0x7fcf68101930 2024-03-19T01:50:41.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.422+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf681a2160 con 0x7fcf68101930 2024-03-19T01:50:41.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.423+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf681a2330 con 0x7fcf68101930 2024-03-19T01:50:41.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.424+0000 7fcf5e7fc640 1 -- 172.21.15.92:0/1844668653 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fcf580041d0 con 0x7fcf68101930 2024-03-19T01:50:41.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.425+0000 7fcf5e7fc640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf540a2800 0x7fcf540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:41.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.425+0000 7fcf6d790640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf540a2800 0x7fcf540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:41.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.425+0000 7fcf5e7fc640 1 -- 172.21.15.92:0/1844668653 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcf580ce030 con 0x7fcf68101930 2024-03-19T01:50:41.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.426+0000 7fcf6d790640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf540a2800 0x7fcf540a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcf6819b1e0 tx=0x7fcf64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:41.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.430+0000 7fcf5e7fc640 1 -- 172.21.15.92:0/1844668653 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcf5809b570 con 0x7fcf68101930 2024-03-19T01:50:41.560 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 01:50:41 smithi092 podman[104532]: 2024-03-19 01:50:41.208928542 +0000 UTC m=+0.373145509 container init 93b782d6d8b8373ea2e3df52471eef7dc082298bb4a39e5c7d4c64d25686ea64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-19T01:50:41.560 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 01:50:41 smithi092 podman[104532]: 2024-03-19 01:50:41.21156205 +0000 UTC m=+0.375779004 container start 93b782d6d8b8373ea2e3df52471eef7dc082298bb4a39e5c7d4c64d25686ea64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0) 2024-03-19T01:50:41.561 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 01:50:41 smithi092 ceph-mds[104551]: -- 172.21.15.92:0/3396992615 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f75b435a40 con 0x55f75b408800 2024-03-19T01:50:41.561 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 01:50:41 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a[104547]: starting mds.a at 2024-03-19T01:50:41.561 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 01:50:41 smithi092 bash[104532]: 93b782d6d8b8373ea2e3df52471eef7dc082298bb4a39e5c7d4c64d25686ea64 2024-03-19T01:50:41.561 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 01:50:41 smithi092 systemd[1]: Started Ceph mds.a for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:41.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.643+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf681a2500 con 0x7fcf68101930 2024-03-19T01:50:41.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.644+0000 7fcf5e7fc640 1 -- 172.21.15.92:0/1844668653 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7fcf5809ae70 con 0x7fcf68101930 2024-03-19T01:50:41.645 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:41.645 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-19T01:50:41.645 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2024-03-19T01:50:41.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf540a2800 msgr2=0x7fcf540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf540a2800 0x7fcf540a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcf6819b1e0 tx=0x7fcf64005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf68101930 msgr2=0x7fcf6819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf68101930 0x7fcf6819a250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800d950 tx=0x7fcf5800de10 comp rx=0 tx=0).stop 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 shutdown_connections 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf540a2800 0x7fcf540a4cc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf681022f0 0x7fcf681a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf68101930 0x7fcf6819a250 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 --2- 172.21.15.92:0/1844668653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf68100740 0x7fcf68199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 >> 172.21.15.92:0/1844668653 conn(0x7fcf680fbcb0 msgr2=0x7fcf680fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 shutdown_connections 2024-03-19T01:50:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:41.646+0000 7fcf6f9f4640 1 -- 172.21.15.92:0/1844668653 wait complete. 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: Deploying daemon mds.b on smithi144 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] up:boot 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: fsmap 11 up:standby 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:42.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:42 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1844668653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:42.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:42.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:42.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:42.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:42.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:42.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:42.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: Deploying daemon mds.b on smithi144 2024-03-19T01:50:42.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] up:boot 2024-03-19T01:50:42.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: fsmap 11 up:standby 2024-03-19T01:50:42.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:50:42.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:42.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:42 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1844668653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-19T01:50:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-19T01:50:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: Deploying daemon mds.b on smithi144 2024-03-19T01:50:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] up:boot 2024-03-19T01:50:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: fsmap 11 up:standby 2024-03-19T01:50:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T01:50:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:42.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:42 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1844668653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:43.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:43.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 -- 172.21.15.92:0/4223381159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 msgr2=0x7f76a80fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 --2- 172.21.15.92:0/4223381159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 0x7f76a80fff50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f769c009930 tx=0x7f769c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 -- 172.21.15.92:0/4223381159 shutdown_connections 2024-03-19T01:50:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 --2- 172.21.15.92:0/4223381159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8103560 0x7f76a8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 --2- 172.21.15.92:0/4223381159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76a8102ba0 0x7f76a8103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 --2- 172.21.15.92:0/4223381159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 0x7f76a80fff50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 -- 172.21.15.92:0/4223381159 >> 172.21.15.92:0/4223381159 conn(0x7f76a80fb440 msgr2=0x7f76a80fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 -- 172.21.15.92:0/4223381159 shutdown_connections 2024-03-19T01:50:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 -- 172.21.15.92:0/4223381159 wait complete. 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.768+0000 7f76addf4640 1 Processor -- start 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76addf4640 1 -- start start 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76addf4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 0x7f76a8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76addf4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76a8102ba0 0x7f76a81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76addf4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8103560 0x7f76a819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76addf4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f76a810d4c0 con 0x7f76a80ffb30 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76addf4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76a810d340 con 0x7f76a8103560 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76addf4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76a810d640 con 0x7f76a8102ba0 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 0x7f76a8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 0x7f76a8197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34866/0 (socket says 172.21.15.92:34866) 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a77fe640 1 -- 172.21.15.92:0/1347603646 learned_addr learned my addr 172.21.15.92:0/1347603646 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a7fff640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8103560 0x7f76a819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a6ffd640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76a8102ba0 0x7f76a81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a77fe640 1 -- 172.21.15.92:0/1347603646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76a8102ba0 msgr2=0x7f76a81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a77fe640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76a8102ba0 0x7f76a81980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a77fe640 1 -- 172.21.15.92:0/1347603646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8103560 msgr2=0x7f76a819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a77fe640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8103560 0x7f76a819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a77fe640 1 -- 172.21.15.92:0/1347603646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a819f780 con 0x7f76a80ffb30 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a6ffd640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76a8102ba0 0x7f76a81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a77fe640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 0x7f76a8197b60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f769c02c7b0 tx=0x7f769c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a4ff9640 1 -- 172.21.15.92:0/1347603646 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c041070 con 0x7f76a80ffb30 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a4ff9640 1 -- 172.21.15.92:0/1347603646 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f769c0054e0 con 0x7f76a80ffb30 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a819fa10 con 0x7f76a80ffb30 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.769+0000 7f76a4ff9640 1 -- 172.21.15.92:0/1347603646 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c034440 con 0x7f76a80ffb30 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.770+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a819ff00 con 0x7f76a80ffb30 2024-03-19T01:50:43.776 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.771+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a8108cd0 con 0x7f76a80ffb30 2024-03-19T01:50:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.772+0000 7f76a4ff9640 1 -- 172.21.15.92:0/1347603646 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f769c044070 con 0x7f76a80ffb30 2024-03-19T01:50:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.772+0000 7f76a4ff9640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f76900a2800 0x7f76900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.772+0000 7f76a6ffd640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f76900a2800 0x7f76900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.772+0000 7f76a4ff9640 1 -- 172.21.15.92:0/1347603646 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f769c0f0230 con 0x7f76a80ffb30 2024-03-19T01:50:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.773+0000 7f76a6ffd640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f76900a2800 0x7f76900a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7698005d00 tx=0x7f7698005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:43.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.777+0000 7f76a4ff9640 1 -- 172.21.15.92:0/1347603646 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f769c0bd770 con 0x7f76a80ffb30 2024-03-19T01:50:43.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.989+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76a802d1f0 con 0x7f76a80ffb30 2024-03-19T01:50:43.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.990+0000 7f76a4ff9640 1 -- 172.21.15.92:0/1347603646 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f769c0bd070 con 0x7f76a80ffb30 2024-03-19T01:50:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:43.990 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-19T01:50:43.991 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 12 2024-03-19T01:50:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f76900a2800 msgr2=0x7f76900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f76900a2800 0x7f76900a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7698005d00 tx=0x7f7698005c70 comp rx=0 tx=0).stop 2024-03-19T01:50:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 msgr2=0x7f76a8197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 0x7f76a8197b60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f769c02c7b0 tx=0x7f769c005dd0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 shutdown_connections 2024-03-19T01:50:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a8103560 0x7f76a819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f76900a2800 0x7f76900a4cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76a8102ba0 0x7f76a81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 --2- 172.21.15.92:0/1347603646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a80ffb30 0x7f76a8197b60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 >> 172.21.15.92:0/1347603646 conn(0x7f76a80fb440 msgr2=0x7f76a8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 shutdown_connections 2024-03-19T01:50:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:43.992+0000 7f76addf4640 1 -- 172.21.15.92:0/1347603646 wait complete. 2024-03-19T01:50:44.004 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 01:50:43 smithi144 podman[97861]: 2024-03-19 01:50:43.867564377 +0000 UTC m=+0.017887610 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T01:50:44.004 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 01:50:44 smithi144 podman[97861]: 2024-03-19 01:50:44.00484711 +0000 UTC m=+0.155170335 container create 7fdf2e218f60fce455691b1296733080b9c4d597be00fd52329e271212de798d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T01:50:44.274 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 01:50:44 smithi144 podman[97861]: 2024-03-19 01:50:44.249154622 +0000 UTC m=+0.399477851 container init 7fdf2e218f60fce455691b1296733080b9c4d597be00fd52329e271212de798d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-19T01:50:44.274 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 01:50:44 smithi144 podman[97861]: 2024-03-19 01:50:44.251742858 +0000 UTC m=+0.402066083 container start 7fdf2e218f60fce455691b1296733080b9c4d597be00fd52329e271212de798d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T01:50:44.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-mon[67199]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:44.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1347603646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:44.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.597 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-mds[97880]: -- 172.21.15.144:0/1052889165 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b03649a40 con 0x555b0361c800 2024-03-19T01:50:44.597 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 01:50:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b[97876]: starting mds.b at 2024-03-19T01:50:44.597 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 01:50:44 smithi144 bash[97861]: 7fdf2e218f60fce455691b1296733080b9c4d597be00fd52329e271212de798d 2024-03-19T01:50:44.597 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 01:50:44 smithi144 systemd[1]: Started Ceph mds.b for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T01:50:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:44 smithi092 ceph-mon[62574]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:44 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1347603646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:44 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:44 smithi159 ceph-mon[67110]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:44 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1347603646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:44.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:44 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:45.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:45 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] up:boot 2024-03-19T01:50:45.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:45 smithi159 ceph-mon[67110]: fsmap 12 up:standby 2024-03-19T01:50:45.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:50:45.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:45 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:50:45.686 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:45 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] up:boot 2024-03-19T01:50:45.686 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:45 smithi092 ceph-mon[62574]: fsmap 12 up:standby 2024-03-19T01:50:45.686 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:50:45.686 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:45 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:50:45.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:45 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] up:boot 2024-03-19T01:50:45.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:45 smithi144 ceph-mon[67199]: fsmap 12 up:standby 2024-03-19T01:50:45.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T01:50:45.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:45 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:50:46.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.114+0000 7f388a9f4640 1 -- 172.21.15.92:0/2880160151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 msgr2=0x7f3884100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.114+0000 7f388a9f4640 1 --2- 172.21.15.92:0/2880160151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 0x7f3884100b60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3878009900 tx=0x7f387802bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.114+0000 7f388a9f4640 1 -- 172.21.15.92:0/2880160151 shutdown_connections 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.114+0000 7f388a9f4640 1 --2- 172.21.15.92:0/2880160151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38841022f0 0x7f3884108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.114+0000 7f388a9f4640 1 --2- 172.21.15.92:0/2880160151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3884101930 0x7f3884101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.114+0000 7f388a9f4640 1 --2- 172.21.15.92:0/2880160151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 0x7f3884100b60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.114+0000 7f388a9f4640 1 -- 172.21.15.92:0/2880160151 >> 172.21.15.92:0/2880160151 conn(0x7f38840fbcb0 msgr2=0x7f38840fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.114+0000 7f388a9f4640 1 -- 172.21.15.92:0/2880160151 shutdown_connections 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.114+0000 7f388a9f4640 1 -- 172.21.15.92:0/2880160151 wait complete. 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f388a9f4640 1 Processor -- start 2024-03-19T01:50:46.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f388a9f4640 1 -- start start 2024-03-19T01:50:46.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f388a9f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 0x7f3884197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f388a9f4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3884101930 0x7f38841980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f388a9f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38841022f0 0x7f388419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f388a9f4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f388410c3b0 con 0x7f3884100740 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f388a9f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f388410c230 con 0x7f38841022f0 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f388a9f4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f388410c530 con 0x7f3884101930 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f3883fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 0x7f3884197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f3883fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 0x7f3884197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34886/0 (socket says 172.21.15.92:34886) 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f3883fff640 1 -- 172.21.15.92:0/3393911715 learned_addr learned my addr 172.21.15.92:0/3393911715 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f38837fe640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3884101930 0x7f38841980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f3888f91640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38841022f0 0x7f388419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f3883fff640 1 -- 172.21.15.92:0/3393911715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3884101930 msgr2=0x7f38841980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f3883fff640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3884101930 0x7f38841980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f3883fff640 1 -- 172.21.15.92:0/3393911715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38841022f0 msgr2=0x7f388419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f3883fff640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38841022f0 0x7f388419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.115+0000 7f3883fff640 1 -- 172.21.15.92:0/3393911715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388419f7d0 con 0x7f3884100740 2024-03-19T01:50:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.116+0000 7f3883fff640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 0x7f3884197bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f387802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.116+0000 7f38817fa640 1 -- 172.21.15.92:0/3393911715 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878041070 con 0x7f3884100740 2024-03-19T01:50:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.116+0000 7f38817fa640 1 -- 172.21.15.92:0/3393911715 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3878004450 con 0x7f3884100740 2024-03-19T01:50:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.116+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388419fa60 con 0x7f3884100740 2024-03-19T01:50:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.116+0000 7f38817fa640 1 -- 172.21.15.92:0/3393911715 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878034440 con 0x7f3884100740 2024-03-19T01:50:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.116+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388419ff50 con 0x7f3884100740 2024-03-19T01:50:46.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.117+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38841a0180 con 0x7f3884100740 2024-03-19T01:50:46.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.118+0000 7f38817fa640 1 -- 172.21.15.92:0/3393911715 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f3878044070 con 0x7f3884100740 2024-03-19T01:50:46.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.119+0000 7f38817fa640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f386c0a2800 0x7f386c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.119+0000 7f38817fa640 1 -- 172.21.15.92:0/3393911715 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f38780f0340 con 0x7f3884100740 2024-03-19T01:50:46.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.119+0000 7f38837fe640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f386c0a2800 0x7f386c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.119+0000 7f38837fe640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f386c0a2800 0x7f386c0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38841990c0 tx=0x7f3874005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:46.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.123+0000 7f38817fa640 1 -- 172.21.15.92:0/3393911715 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f38780bd910 con 0x7f3884100740 2024-03-19T01:50:46.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.380+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38841a0350 con 0x7f3884100740 2024-03-19T01:50:46.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.381+0000 7f38817fa640 1 -- 172.21.15.92:0/3393911715 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7f38780bd210 con 0x7f3884100740 2024-03-19T01:50:46.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:46.382 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/380126062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":380126062},{"type":"v1","addr":"172.21.15.144:6839","nonce":380126062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-19T01:50:46.382 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 13 2024-03-19T01:50:46.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f386c0a2800 msgr2=0x7f386c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f386c0a2800 0x7f386c0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38841990c0 tx=0x7f3874005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:46.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 msgr2=0x7f3884197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 0x7f3884197bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f387802c640 comp rx=0 tx=0).stop 2024-03-19T01:50:46.384 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 shutdown_connections 2024-03-19T01:50:46.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38841022f0 0x7f388419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f386c0a2800 0x7f386c0a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3884101930 0x7f38841980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 --2- 172.21.15.92:0/3393911715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3884100740 0x7f3884197bb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 >> 172.21.15.92:0/3393911715 conn(0x7f38840fbcb0 msgr2=0x7f38840fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:46.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 shutdown_connections 2024-03-19T01:50:46.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.383+0000 7f388a9f4640 1 -- 172.21.15.92:0/3393911715 wait complete. 2024-03-19T01:50:46.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:50:46.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 -- 172.21.15.92:0/2205314613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 msgr2=0x7f2614109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 --2- 172.21.15.92:0/2205314613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 0x7f2614109d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2604009930 tx=0x7f260402bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 -- 172.21.15.92:0/2205314613 shutdown_connections 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 --2- 172.21.15.92:0/2205314613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 0x7f2614109d50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 --2- 172.21.15.92:0/2205314613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2614102b00 0x7f2614102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 --2- 172.21.15.92:0/2205314613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26140ffb30 0x7f26140fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 -- 172.21.15.92:0/2205314613 >> 172.21.15.92:0/2205314613 conn(0x7f26140fb460 msgr2=0x7f26140fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 -- 172.21.15.92:0/2205314613 shutdown_connections 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 -- 172.21.15.92:0/2205314613 wait complete. 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 Processor -- start 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 -- start start 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.498+0000 7f261b019640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26140ffb30 0x7f2614197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f261b019640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2614102b00 0x7f2614198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.501 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f261b019640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 0x7f261419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f261b019640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f261410d500 con 0x7f26141034c0 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f261b019640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f261410d380 con 0x7f2614102b00 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f261b019640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f261410d680 con 0x7f26140ffb30 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f26195b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 0x7f261419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f26195b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 0x7f261419f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34902/0 (socket says 172.21.15.92:34902) 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f26195b6640 1 -- 172.21.15.92:0/779966460 learned_addr learned my addr 172.21.15.92:0/779966460 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f2613fff640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2614102b00 0x7f2614198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f2618db5640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26140ffb30 0x7f2614197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f26195b6640 1 -- 172.21.15.92:0/779966460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26140ffb30 msgr2=0x7f2614197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f26195b6640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26140ffb30 0x7f2614197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f26195b6640 1 -- 172.21.15.92:0/779966460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2614102b00 msgr2=0x7f2614198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f26195b6640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2614102b00 0x7f2614198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f26195b6640 1 -- 172.21.15.92:0/779966460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261419f760 con 0x7f26141034c0 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f2618db5640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26140ffb30 0x7f2614197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f26195b6640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 0x7f261419f0f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f260402c8e0 tx=0x7f2604004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f2611ffb640 1 -- 172.21.15.92:0/779966460 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604041070 con 0x7f26141034c0 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261419f9f0 con 0x7f26141034c0 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f2611ffb640 1 -- 172.21.15.92:0/779966460 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26040054e0 con 0x7f26141034c0 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f2611ffb640 1 -- 172.21.15.92:0/779966460 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26040346a0 con 0x7f26141034c0 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.499+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261419ff30 con 0x7f26141034c0 2024-03-19T01:50:46.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.501+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2614108cb0 con 0x7f26141034c0 2024-03-19T01:50:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.501+0000 7f2611ffb640 1 -- 172.21.15.92:0/779966460 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f2604044070 con 0x7f26141034c0 2024-03-19T01:50:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.502+0000 7f2611ffb640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f26000a2800 0x7f26000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.502+0000 7f2618db5640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f26000a2800 0x7f26000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.502+0000 7f2611ffb640 1 -- 172.21.15.92:0/779966460 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f26040f0310 con 0x7f26141034c0 2024-03-19T01:50:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.503+0000 7f2618db5640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f26000a2800 0x7f26000a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f260c005c40 tx=0x7f260c00a430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.507+0000 7f2611ffb640 1 -- 172.21.15.92:0/779966460 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f26040bd850 con 0x7f26141034c0 2024-03-19T01:50:46.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.695+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f261402d3a0 con 0x7f26141034c0 2024-03-19T01:50:46.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.696+0000 7f2611ffb640 1 -- 172.21.15.92:0/779966460 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18493 (secure 0 0 0) 0x7f26040dd0d0 con 0x7f26141034c0 2024-03-19T01:50:46.697 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:46.697 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":81,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:50:00.507820+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:50:46.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f26000a2800 msgr2=0x7f26000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f26000a2800 0x7f26000a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f260c005c40 tx=0x7f260c00a430 comp rx=0 tx=0).stop 2024-03-19T01:50:46.699 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 msgr2=0x7f261419f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 0x7f261419f0f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f260402c8e0 tx=0x7f2604004040 comp rx=0 tx=0).stop 2024-03-19T01:50:46.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 shutdown_connections 2024-03-19T01:50:46.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26141034c0 0x7f261419f0f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2614102b00 0x7f2614198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f26000a2800 0x7f26000a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 --2- 172.21.15.92:0/779966460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26140ffb30 0x7f2614197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 >> 172.21.15.92:0/779966460 conn(0x7f26140fb460 msgr2=0x7f2614105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:46.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 shutdown_connections 2024-03-19T01:50:46.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.699+0000 7f261b019640 1 -- 172.21.15.92:0/779966460 wait complete. 2024-03-19T01:50:46.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:46.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.709 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:46.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:46.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3393911715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:46.710 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:46 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:46:07.695753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:50:46.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 -- 172.21.15.92:0/2844994900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b01022d0 msgr2=0x7fb5b0108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 --2- 172.21.15.92:0/2844994900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b01022d0 0x7fb5b0108b60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009ca0 tx=0x7fb5a802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 -- 172.21.15.92:0/2844994900 shutdown_connections 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 --2- 172.21.15.92:0/2844994900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b01022d0 0x7fb5b0108b60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 --2- 172.21.15.92:0/2844994900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0101910 0x7fb5b0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 --2- 172.21.15.92:0/2844994900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b0100720 0x7fb5b0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 -- 172.21.15.92:0/2844994900 >> 172.21.15.92:0/2844994900 conn(0x7fb5b00fbcd0 msgr2=0x7fb5b00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 -- 172.21.15.92:0/2844994900 shutdown_connections 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 -- 172.21.15.92:0/2844994900 wait complete. 2024-03-19T01:50:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.819+0000 7fb5b551b640 1 Processor -- start 2024-03-19T01:50:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5b551b640 1 -- start start 2024-03-19T01:50:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5b551b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100720 0x7fb5b01969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5b551b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b0101910 0x7fb5b0196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5b551b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b01022d0 0x7fb5b019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5b551b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b010c380 con 0x7fb5b0101910 2024-03-19T01:50:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5b551b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b010c200 con 0x7fb5b01022d0 2024-03-19T01:50:46.823 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5b551b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b010c500 con 0x7fb5b0100720 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5ae7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b0101910 0x7fb5b0196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5ae7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b0101910 0x7fb5b0196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34916/0 (socket says 172.21.15.92:34916) 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5ae7fc640 1 -- 172.21.15.92:0/4288607014 learned_addr learned my addr 172.21.15.92:0/4288607014 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5aeffd640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100720 0x7fb5b01969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5af7fe640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b01022d0 0x7fb5b019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5ae7fc640 1 -- 172.21.15.92:0/4288607014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100720 msgr2=0x7fb5b01969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5ae7fc640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100720 0x7fb5b01969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5ae7fc640 1 -- 172.21.15.92:0/4288607014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b01022d0 msgr2=0x7fb5b019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5ae7fc640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b01022d0 0x7fb5b019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5ae7fc640 1 -- 172.21.15.92:0/4288607014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b019e5e0 con 0x7fb5b0101910 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5af7fe640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b01022d0 0x7fb5b019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.820+0000 7fb5ae7fc640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b0101910 0x7fb5b0196f00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400d950 tx=0x7fb5a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.821+0000 7fb597fff640 1 -- 172.21.15.92:0/4288607014 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4017070 con 0x7fb5b0101910 2024-03-19T01:50:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.821+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b019e8d0 con 0x7fb5b0101910 2024-03-19T01:50:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.821+0000 7fb597fff640 1 -- 172.21.15.92:0/4288607014 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a4009c30 con 0x7fb5b0101910 2024-03-19T01:50:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.821+0000 7fb597fff640 1 -- 172.21.15.92:0/4288607014 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400f6b0 con 0x7fb5b0101910 2024-03-19T01:50:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.821+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b019ee70 con 0x7fb5b0101910 2024-03-19T01:50:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.822+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b0107a40 con 0x7fb5b0101910 2024-03-19T01:50:46.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.823+0000 7fb597fff640 1 -- 172.21.15.92:0/4288607014 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb5a40109d0 con 0x7fb5b0101910 2024-03-19T01:50:46.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.824+0000 7fb597fff640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb5980a2800 0x7fb5980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:46.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.824+0000 7fb597fff640 1 -- 172.21.15.92:0/4288607014 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb5a40ce230 con 0x7fb5b0101910 2024-03-19T01:50:46.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.824+0000 7fb5aeffd640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb5980a2800 0x7fb5980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:46.826 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.824+0000 7fb5aeffd640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb5980a2800 0x7fb5980a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0101770 tx=0x7fb5a000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:46.828 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:46.829+0000 7fb597fff640 1 -- 172.21.15.92:0/4288607014 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb5a4009de0 con 0x7fb5b0101910 2024-03-19T01:50:46.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:46.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:46.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:46.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3393911715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:46.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:46 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:46.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:46.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:46.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3393911715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:46.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:46 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:47.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.015+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5b002d400 con 0x7fb5b0101910 2024-03-19T01:50:47.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.016+0000 7fb597fff640 1 -- 172.21.15.92:0/4288607014 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fb5a409b770 con 0x7fb5b0101910 2024-03-19T01:50:47.016 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-03-19T01:50:47.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb5980a2800 msgr2=0x7fb5980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb5980a2800 0x7fb5980a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0101770 tx=0x7fb5a000b390 comp rx=0 tx=0).stop 2024-03-19T01:50:47.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b0101910 msgr2=0x7fb5b0196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b0101910 0x7fb5b0196f00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400d950 tx=0x7fb5a400de10 comp rx=0 tx=0).stop 2024-03-19T01:50:47.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 shutdown_connections 2024-03-19T01:50:47.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5b01022d0 0x7fb5b019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5b0101910 0x7fb5b0196f00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb5980a2800 0x7fb5980a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 --2- 172.21.15.92:0/4288607014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100720 0x7fb5b01969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 >> 172.21.15.92:0/4288607014 conn(0x7fb5b00fbcd0 msgr2=0x7fb5b0104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:47.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 shutdown_connections 2024-03-19T01:50:47.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.018+0000 7fb5b551b640 1 -- 172.21.15.92:0/4288607014 wait complete. 2024-03-19T01:50:47.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-19T01:50:47.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.141+0000 7f53a26b2640 1 -- 172.21.15.92:0/3635215881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c100740 msgr2=0x7f539c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.141+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3635215881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c100740 0x7f539c100b60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5390009900 tx=0x7f539002bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:47.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.141+0000 7f53a26b2640 1 -- 172.21.15.92:0/3635215881 shutdown_connections 2024-03-19T01:50:47.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.141+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3635215881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c1022f0 0x7f539c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.141+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3635215881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c101930 0x7f539c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.141+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3635215881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c100740 0x7f539c100b60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.141+0000 7f53a26b2640 1 -- 172.21.15.92:0/3635215881 >> 172.21.15.92:0/3635215881 conn(0x7f539c0fbcb0 msgr2=0x7f539c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.141+0000 7f53a26b2640 1 -- 172.21.15.92:0/3635215881 shutdown_connections 2024-03-19T01:50:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.141+0000 7f53a26b2640 1 -- 172.21.15.92:0/3635215881 wait complete. 2024-03-19T01:50:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f53a26b2640 1 Processor -- start 2024-03-19T01:50:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f53a26b2640 1 -- start start 2024-03-19T01:50:47.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f53a26b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100740 0x7f539c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f53a26b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c101930 0x7f539c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f53a26b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c1022f0 0x7f539c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f53a26b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f539c10c430 con 0x7f539c101930 2024-03-19T01:50:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f53a26b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f539c10c2b0 con 0x7f539c1022f0 2024-03-19T01:50:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f53a26b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f539c10c5b0 con 0x7f539c100740 2024-03-19T01:50:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c101930 0x7f539c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c101930 0x7f539c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34922/0 (socket says 172.21.15.92:34922) 2024-03-19T01:50:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.142+0000 7f539b7fe640 1 -- 172.21.15.92:0/3746694548 learned_addr learned my addr 172.21.15.92:0/3746694548 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f53a0c4f640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c1022f0 0x7f539c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f539bfff640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100740 0x7f539c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f539b7fe640 1 -- 172.21.15.92:0/3746694548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100740 msgr2=0x7f539c197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f539b7fe640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100740 0x7f539c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f539b7fe640 1 -- 172.21.15.92:0/3746694548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c1022f0 msgr2=0x7f539c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f539b7fe640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c1022f0 0x7f539c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f539b7fe640 1 -- 172.21.15.92:0/3746694548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c19f830 con 0x7f539c101930 2024-03-19T01:50:47.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f53a0c4f640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c1022f0 0x7f539c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:50:47.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f539b7fe640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c101930 0x7f539c1981b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f538c00d770 tx=0x7f538c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:47.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f53997fa640 1 -- 172.21.15.92:0/3746694548 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c017070 con 0x7f539c101930 2024-03-19T01:50:47.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f53997fa640 1 -- 172.21.15.92:0/3746694548 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f538c005500 con 0x7f539c101930 2024-03-19T01:50:47.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c19fb20 con 0x7f539c101930 2024-03-19T01:50:47.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f53997fa640 1 -- 172.21.15.92:0/3746694548 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c004e00 con 0x7f539c101930 2024-03-19T01:50:47.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.143+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1a0020 con 0x7f539c101930 2024-03-19T01:50:47.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.144+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c1a01f0 con 0x7f539c101930 2024-03-19T01:50:47.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.145+0000 7f53997fa640 1 -- 172.21.15.92:0/3746694548 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f538c004fa0 con 0x7f539c101930 2024-03-19T01:50:47.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.146+0000 7f53997fa640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53840a2800 0x7f53840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:47.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.146+0000 7f539bfff640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53840a2800 0x7f53840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:47.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.146+0000 7f53997fa640 1 -- 172.21.15.92:0/3746694548 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f538c0ce580 con 0x7f539c101930 2024-03-19T01:50:47.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.146+0000 7f539bfff640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53840a2800 0x7f53840a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5390009230 tx=0x7f5390005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:47.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.150+0000 7f53997fa640 1 -- 172.21.15.92:0/3746694548 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f538c09bac0 con 0x7f539c101930 2024-03-19T01:50:47.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.331+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f539c1a03c0 con 0x7f539c101930 2024-03-19T01:50:47.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.331+0000 7f53997fa640 1 -- 172.21.15.92:0/3746694548 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f538c09b3c0 con 0x7f539c101930 2024-03-19T01:50:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:47.332 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2024-03-19T01:50:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T01:50:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2024-03-19T01:50:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2024-03-19T01:50:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T01:50:47.333 INFO:teuthology.orchestra.run.smithi092.stdout:] 2024-03-19T01:50:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53840a2800 msgr2=0x7f53840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53840a2800 0x7f53840a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5390009230 tx=0x7f5390005fd0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c101930 msgr2=0x7f539c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c101930 0x7f539c1981b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f538c00d770 tx=0x7f538c00dc30 comp rx=0 tx=0).stop 2024-03-19T01:50:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 shutdown_connections 2024-03-19T01:50:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539c1022f0 0x7f539c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f539c101930 0x7f539c1981b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f53840a2800 0x7f53840a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 --2- 172.21.15.92:0/3746694548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100740 0x7f539c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 >> 172.21.15.92:0/3746694548 conn(0x7f539c0fbcb0 msgr2=0x7f539c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:47.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 shutdown_connections 2024-03-19T01:50:47.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.334+0000 7f53a26b2640 1 -- 172.21.15.92:0/3746694548 wait complete. 2024-03-19T01:50:47.345 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-19T01:50:47.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-19T01:50:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 -- 172.21.15.92:0/1959649856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc101910 msgr2=0x7fc4fc101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 --2- 172.21.15.92:0/1959649856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc101910 0x7fc4fc101d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009900 tx=0x7fc4f802bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 -- 172.21.15.92:0/1959649856 shutdown_connections 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 --2- 172.21.15.92:0/1959649856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4fc1022d0 0x7fc4fc108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 --2- 172.21.15.92:0/1959649856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc101910 0x7fc4fc101d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 --2- 172.21.15.92:0/1959649856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4fc100720 0x7fc4fc100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 -- 172.21.15.92:0/1959649856 >> 172.21.15.92:0/1959649856 conn(0x7fc4fc0fbcb0 msgr2=0x7fc4fc0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 -- 172.21.15.92:0/1959649856 shutdown_connections 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 -- 172.21.15.92:0/1959649856 wait complete. 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 Processor -- start 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.456+0000 7fc5044ea640 1 -- start start 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc5044ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4fc100720 0x7fc4fc10b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc5044ea640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4fc101910 0x7fc4fc1096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc5044ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc1022d0 0x7fc4fc109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc5044ea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10d520 con 0x7fc4fc1022d0 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc5044ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10d3a0 con 0x7fc4fc100720 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc5044ea640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc10d6a0 con 0x7fc4fc101910 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502a87640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc1022d0 0x7fc4fc109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502a87640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc1022d0 0x7fc4fc109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34932/0 (socket says 172.21.15.92:34932) 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502a87640 1 -- 172.21.15.92:0/2682548864 learned_addr learned my addr 172.21.15.92:0/2682548864 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502286640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4fc100720 0x7fc4fc10b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc501a85640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4fc101910 0x7fc4fc1096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502a87640 1 -- 172.21.15.92:0/2682548864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4fc101910 msgr2=0x7fc4fc1096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502a87640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4fc101910 0x7fc4fc1096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502a87640 1 -- 172.21.15.92:0/2682548864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4fc100720 msgr2=0x7fc4fc10b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502a87640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4fc100720 0x7fc4fc10b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502a87640 1 -- 172.21.15.92:0/2682548864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc10a4b0 con 0x7fc4fc1022d0 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502286640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4fc100720 0x7fc4fc10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc502a87640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc1022d0 0x7fc4fc109c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400b970 tx=0x7fc4f400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc4f37fe640 1 -- 172.21.15.92:0/2682548864 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4019070 con 0x7fc4fc1022d0 2024-03-19T01:50:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4fc1a1cf0 con 0x7fc4fc1022d0 2024-03-19T01:50:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc4f37fe640 1 -- 172.21.15.92:0/2682548864 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f400cb40 con 0x7fc4fc1022d0 2024-03-19T01:50:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc4f37fe640 1 -- 172.21.15.92:0/2682548864 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4007650 con 0x7fc4fc1022d0 2024-03-19T01:50:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.457+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4fc1a2190 con 0x7fc4fc1022d0 2024-03-19T01:50:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.458+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4fc107b60 con 0x7fc4fc1022d0 2024-03-19T01:50:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.459+0000 7fc4f37fe640 1 -- 172.21.15.92:0/2682548864 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc4f401c070 con 0x7fc4fc1022d0 2024-03-19T01:50:47.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.460+0000 7fc4f37fe640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4e80a2800 0x7fc4e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:47.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.460+0000 7fc502286640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4e80a2800 0x7fc4e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:47.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.460+0000 7fc4f37fe640 1 -- 172.21.15.92:0/2682548864 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc4f40ce110 con 0x7fc4fc1022d0 2024-03-19T01:50:47.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.460+0000 7fc502286640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4e80a2800 0x7fc4e80a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc101770 tx=0x7fc4ec005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:47.463 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.463+0000 7fc4f37fe640 1 -- 172.21.15.92:0/2682548864 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc4f409b650 con 0x7fc4fc1022d0 2024-03-19T01:50:47.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:47 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:50:47.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:47 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/779966460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:47.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:47 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4288607014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:50:47.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:47 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3746694548' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-19T01:50:47.607 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:47 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:50:47.607 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:47 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/779966460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:47.608 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:47 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4288607014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:50:47.608 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:47 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3746694548' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-19T01:50:47.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:47.646+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc4fc02d290 con 0x7fc4fc1022d0 2024-03-19T01:50:47.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:47 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:50:47.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:47 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/779966460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:47.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:47 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4288607014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:50:47.706 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:47 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3746694548' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-19T01:50:48.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.523+0000 7fc4f37fe640 1 -- 172.21.15.92:0/2682548864 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fc4f409af50 con 0x7fc4fc1022d0 2024-03-19T01:50:48.523 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2024-03-19T01:50:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4e80a2800 msgr2=0x7fc4e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:48.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4e80a2800 0x7fc4e80a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc101770 tx=0x7fc4ec005e70 comp rx=0 tx=0).stop 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc1022d0 msgr2=0x7fc4fc109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc1022d0 0x7fc4fc109c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400b970 tx=0x7fc4f400be30 comp rx=0 tx=0).stop 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 shutdown_connections 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4fc1022d0 0x7fc4fc109c20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4e80a2800 0x7fc4e80a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4fc101910 0x7fc4fc1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 --2- 172.21.15.92:0/2682548864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4fc100720 0x7fc4fc10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 >> 172.21.15.92:0/2682548864 conn(0x7fc4fc0fbcb0 msgr2=0x7fc4fc104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 shutdown_connections 2024-03-19T01:50:48.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.525+0000 7fc5044ea640 1 -- 172.21.15.92:0/2682548864 wait complete. 2024-03-19T01:50:48.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-19T01:50:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.647+0000 7f22ab73b640 1 -- 172.21.15.92:0/825568545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102ad0 msgr2=0x7f22a4102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.647+0000 7f22ab73b640 1 --2- 172.21.15.92:0/825568545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102ad0 0x7f22a4102f50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2294009930 tx=0x7f229402bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.647+0000 7f22ab73b640 1 -- 172.21.15.92:0/825568545 shutdown_connections 2024-03-19T01:50:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.647+0000 7f22ab73b640 1 --2- 172.21.15.92:0/825568545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22a4103490 0x7f22a4109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.647+0000 7f22ab73b640 1 --2- 172.21.15.92:0/825568545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4102ad0 0x7f22a4102f50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.647+0000 7f22ab73b640 1 --2- 172.21.15.92:0/825568545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a40ffb00 0x7f22a40fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.647+0000 7f22ab73b640 1 -- 172.21.15.92:0/825568545 >> 172.21.15.92:0/825568545 conn(0x7f22a40fb410 msgr2=0x7f22a40fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.647+0000 7f22ab73b640 1 -- 172.21.15.92:0/825568545 shutdown_connections 2024-03-19T01:50:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.647+0000 7f22ab73b640 1 -- 172.21.15.92:0/825568545 wait complete. 2024-03-19T01:50:48.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22ab73b640 1 Processor -- start 2024-03-19T01:50:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22ab73b640 1 -- start start 2024-03-19T01:50:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22ab73b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22a40ffb00 0x7f22a4197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22ab73b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a4102ad0 0x7f22a41980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22ab73b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4103490 0x7f22a419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22ab73b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22a410d4e0 con 0x7f22a4103490 2024-03-19T01:50:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22ab73b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22a410d360 con 0x7f22a4102ad0 2024-03-19T01:50:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22ab73b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22a410d660 con 0x7f22a40ffb00 2024-03-19T01:50:48.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a9cd8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4103490 0x7f22a419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a9cd8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4103490 0x7f22a419f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34952/0 (socket says 172.21.15.92:34952) 2024-03-19T01:50:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a9cd8640 1 -- 172.21.15.92:0/2007629157 learned_addr learned my addr 172.21.15.92:0/2007629157 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a94d7640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22a40ffb00 0x7f22a4197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a8cd6640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a4102ad0 0x7f22a41980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a9cd8640 1 -- 172.21.15.92:0/2007629157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22a40ffb00 msgr2=0x7f22a4197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a9cd8640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22a40ffb00 0x7f22a4197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a9cd8640 1 -- 172.21.15.92:0/2007629157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a4102ad0 msgr2=0x7f22a41980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:48.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a9cd8640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a4102ad0 0x7f22a41980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:48.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a9cd8640 1 -- 172.21.15.92:0/2007629157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a419f790 con 0x7f22a4103490 2024-03-19T01:50:48.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a8cd6640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a4102ad0 0x7f22a41980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:48.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f22a9cd8640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4103490 0x7f22a419f120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f229c00eaa0 tx=0x7f229c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:48.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.648+0000 7f229a7fc640 1 -- 172.21.15.92:0/2007629157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c00cdb0 con 0x7f22a4103490 2024-03-19T01:50:48.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.649+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a419fa80 con 0x7f22a4103490 2024-03-19T01:50:48.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.649+0000 7f229a7fc640 1 -- 172.21.15.92:0/2007629157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229c005020 con 0x7f22a4103490 2024-03-19T01:50:48.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.649+0000 7f229a7fc640 1 -- 172.21.15.92:0/2007629157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c0116d0 con 0x7f22a4103490 2024-03-19T01:50:48.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.649+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a419ffa0 con 0x7f22a4103490 2024-03-19T01:50:48.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.650+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a4108ca0 con 0x7f22a4103490 2024-03-19T01:50:48.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.651+0000 7f229a7fc640 1 -- 172.21.15.92:0/2007629157 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f229c0054d0 con 0x7f22a4103490 2024-03-19T01:50:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.651+0000 7f229a7fc640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22900a2800 0x7f22900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.651+0000 7f22a94d7640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22900a2800 0x7f22900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.652+0000 7f229a7fc640 1 -- 172.21.15.92:0/2007629157 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f229c0cecd0 con 0x7f22a4103490 2024-03-19T01:50:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.652+0000 7f22a94d7640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22900a2800 0x7f22900a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005fd0 tx=0x7f22a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.655+0000 7f229a7fc640 1 -- 172.21.15.92:0/2007629157 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f229c016030 con 0x7f22a4103490 2024-03-19T01:50:48.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:48.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2682548864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T01:50:48.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:48.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:48.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:48.831+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f22a4062420 con 0x7f22a4103490 2024-03-19T01:50:48.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2682548864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:48.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2682548864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-19T01:50:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:48.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:50:48.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:50:48.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:50:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.528+0000 7f229a7fc640 1 -- 172.21.15.92:0/2007629157 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f229c09c0a0 con 0x7f22a4103490 2024-03-19T01:50:49.529 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22900a2800 msgr2=0x7f22900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22900a2800 0x7f22900a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005fd0 tx=0x7f22a000b410 comp rx=0 tx=0).stop 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4103490 msgr2=0x7f22a419f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4103490 0x7f22a419f120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f229c00eaa0 tx=0x7f229c00ef60 comp rx=0 tx=0).stop 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 shutdown_connections 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22a4103490 0x7f22a419f120 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22a4102ad0 0x7f22a41980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22900a2800 0x7f22900a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 --2- 172.21.15.92:0/2007629157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22a40ffb00 0x7f22a4197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 >> 172.21.15.92:0/2007629157 conn(0x7f22a40fb410 msgr2=0x7f22a4105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 shutdown_connections 2024-03-19T01:50:49.532 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.531+0000 7f22ab73b640 1 -- 172.21.15.92:0/2007629157 wait complete. 2024-03-19T01:50:49.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-19T01:50:49.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.658+0000 7f073473b640 1 -- 172.21.15.92:0/812932604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 msgr2=0x7f072c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:49.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.658+0000 7f073473b640 1 --2- 172.21.15.92:0/812932604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 0x7f072c101d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0728009900 tx=0x7f072802bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:49.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.658+0000 7f073473b640 1 -- 172.21.15.92:0/812932604 shutdown_connections 2024-03-19T01:50:49.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.658+0000 7f073473b640 1 --2- 172.21.15.92:0/812932604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f072c1022d0 0x7f072c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:49.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.658+0000 7f073473b640 1 --2- 172.21.15.92:0/812932604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 0x7f072c101d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:49.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.658+0000 7f073473b640 1 --2- 172.21.15.92:0/812932604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072c100720 0x7f072c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:49.661 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.658+0000 7f073473b640 1 -- 172.21.15.92:0/812932604 >> 172.21.15.92:0/812932604 conn(0x7f072c0fbcb0 msgr2=0x7f072c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 -- 172.21.15.92:0/812932604 shutdown_connections 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 -- 172.21.15.92:0/812932604 wait complete. 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 Processor -- start 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 -- start start 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072c100720 0x7f072c10b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 0x7f072c1096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f072c1022d0 0x7f072c109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10d520 con 0x7f072c101910 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10d3a0 con 0x7f072c1022d0 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f073473b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10d6a0 con 0x7f072c100720 2024-03-19T01:50:49.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f0731cd6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 0x7f072c1096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f0731cd6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 0x7f072c1096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34974/0 (socket says 172.21.15.92:34974) 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.659+0000 7f0731cd6640 1 -- 172.21.15.92:0/660756190 learned_addr learned my addr 172.21.15.92:0/660756190 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f07324d7640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072c100720 0x7f072c10b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f0731cd6640 1 -- 172.21.15.92:0/660756190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072c100720 msgr2=0x7f072c10b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f0731cd6640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072c100720 0x7f072c10b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f0732cd8640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f072c1022d0 0x7f072c109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f0731cd6640 1 -- 172.21.15.92:0/660756190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f072c1022d0 msgr2=0x7f072c109c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f0731cd6640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f072c1022d0 0x7f072c109c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f0731cd6640 1 -- 172.21.15.92:0/660756190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c10a4b0 con 0x7f072c101910 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f0732cd8640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f072c1022d0 0x7f072c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f0731cd6640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 0x7f072c1096e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0728009230 tx=0x7f0728005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f071f7fe640 1 -- 172.21.15.92:0/660756190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728041070 con 0x7f072c101910 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f071f7fe640 1 -- 172.21.15.92:0/660756190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07280054e0 con 0x7f072c101910 2024-03-19T01:50:49.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072c1a1c90 con 0x7f072c101910 2024-03-19T01:50:49.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f071f7fe640 1 -- 172.21.15.92:0/660756190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728034440 con 0x7f072c101910 2024-03-19T01:50:49.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.660+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072c1a2190 con 0x7f072c101910 2024-03-19T01:50:49.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.661+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f072c107b60 con 0x7f072c101910 2024-03-19T01:50:49.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.662+0000 7f071f7fe640 1 -- 172.21.15.92:0/660756190 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0728044070 con 0x7f072c101910 2024-03-19T01:50:49.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.663+0000 7f071f7fe640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07180a2800 0x7f07180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:49.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.663+0000 7f07324d7640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07180a2800 0x7f07180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:49.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.663+0000 7f071f7fe640 1 -- 172.21.15.92:0/660756190 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f07280f0aa0 con 0x7f072c101910 2024-03-19T01:50:49.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.663+0000 7f07324d7640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07180a2800 0x7f07180a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0720005fd0 tx=0x7f0720005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:49.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.667+0000 7f071f7fe640 1 -- 172.21.15.92:0/660756190 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f07280bdcf0 con 0x7f072c101910 2024-03-19T01:50:49.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:49 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2682548864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-19T01:50:49.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:49 smithi092 ceph-mon[62574]: osdmap e82: 12 total, 12 up, 12 in 2024-03-19T01:50:49.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:49 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2007629157' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T01:50:49.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:49 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2682548864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-19T01:50:49.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:49 smithi144 ceph-mon[67199]: osdmap e82: 12 total, 12 up, 12 in 2024-03-19T01:50:49.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:49 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2007629157' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T01:50:49.879 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:49.879+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f072c062490 con 0x7f072c101910 2024-03-19T01:50:49.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:49 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2682548864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-19T01:50:49.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:49 smithi159 ceph-mon[67110]: osdmap e82: 12 total, 12 up, 12 in 2024-03-19T01:50:49.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:49 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2007629157' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-19T01:50:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.530+0000 7f071f7fe640 1 -- 172.21.15.92:0/660756190 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f07280bd5f0 con 0x7f072c101910 2024-03-19T01:50:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 2 and data pool 3 2024-03-19T01:50:50.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07180a2800 msgr2=0x7f07180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07180a2800 0x7f07180a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0720005fd0 tx=0x7f0720005850 comp rx=0 tx=0).stop 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 msgr2=0x7f072c1096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 0x7f072c1096e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0728009230 tx=0x7f0728005dd0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 shutdown_connections 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f072c1022d0 0x7f072c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f072c101910 0x7f072c1096e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07180a2800 0x7f07180a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 --2- 172.21.15.92:0/660756190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072c100720 0x7f072c10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 >> 172.21.15.92:0/660756190 conn(0x7f072c0fbcb0 msgr2=0x7f072c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 shutdown_connections 2024-03-19T01:50:50.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.533+0000 7f073473b640 1 -- 172.21.15.92:0/660756190 wait complete. 2024-03-19T01:50:50.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:50:50.662 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/1246710177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c40ffb30 msgr2=0x7fe2c40fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/1246710177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c40ffb30 0x7fe2c40fff50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009930 tx=0x7fe2b402bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/1246710177 shutdown_connections 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/1246710177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2c41034c0 0x7fe2c4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/1246710177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2c4102b00 0x7fe2c4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/1246710177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c40ffb30 0x7fe2c40fff50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/1246710177 >> 172.21.15.92:0/1246710177 conn(0x7fe2c40fb460 msgr2=0x7fe2c40fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/1246710177 shutdown_connections 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/1246710177 wait complete. 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 Processor -- start 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.661+0000 7fe2c9fb3640 1 -- start start 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c9fb3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2c40ffb30 0x7fe2c4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c9fb3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2c4102b00 0x7fe2c4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c9fb3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c41034c0 0x7fe2c419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c9fb3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c410d500 con 0x7fe2c41034c0 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c9fb3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c410d380 con 0x7fe2c4102b00 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c9fb3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c410d680 con 0x7fe2c40ffb30 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c41034c0 0x7fe2c419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c41034c0 0x7fe2c419f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34988/0 (socket says 172.21.15.92:34988) 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c3fff640 1 -- 172.21.15.92:0/3277605814 learned_addr learned my addr 172.21.15.92:0/3277605814 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c2ffd640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2c4102b00 0x7fe2c4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c37fe640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2c40ffb30 0x7fe2c4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c3fff640 1 -- 172.21.15.92:0/3277605814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2c40ffb30 msgr2=0x7fe2c4197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c3fff640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2c40ffb30 0x7fe2c4197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c3fff640 1 -- 172.21.15.92:0/3277605814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2c4102b00 msgr2=0x7fe2c4198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c3fff640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2c4102b00 0x7fe2c4198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c3fff640 1 -- 172.21.15.92:0/3277605814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c419f7e0 con 0x7fe2c41034c0 2024-03-19T01:50:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c37fe640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2c40ffb30 0x7fe2c4197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c3fff640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c41034c0 0x7fe2c419f170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00eaa0 tx=0x7fe2bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c0ff9640 1 -- 172.21.15.92:0/3277605814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc00cdb0 con 0x7fe2c41034c0 2024-03-19T01:50:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c0ff9640 1 -- 172.21.15.92:0/3277605814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2bc005020 con 0x7fe2c41034c0 2024-03-19T01:50:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c419fad0 con 0x7fe2c41034c0 2024-03-19T01:50:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c0ff9640 1 -- 172.21.15.92:0/3277605814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc0116b0 con 0x7fe2c41034c0 2024-03-19T01:50:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.662+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c41a0020 con 0x7fe2c41034c0 2024-03-19T01:50:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.663+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c4108d00 con 0x7fe2c41034c0 2024-03-19T01:50:50.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.664+0000 7fe2c0ff9640 1 -- 172.21.15.92:0/3277605814 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe2bc005560 con 0x7fe2c41034c0 2024-03-19T01:50:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.665+0000 7fe2c0ff9640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe2ac0a2800 0x7fe2ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.665+0000 7fe2c37fe640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe2ac0a2800 0x7fe2ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.665+0000 7fe2c0ff9640 1 -- 172.21.15.92:0/3277605814 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fe2bc0ceb70 con 0x7fe2c41034c0 2024-03-19T01:50:50.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.665+0000 7fe2c37fe640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe2ac0a2800 0x7fe2ac0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe2b402c7b0 tx=0x7fe2b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:50.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.669+0000 7fe2c0ff9640 1 -- 172.21.15.92:0/3277605814 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe2bc016030 con 0x7fe2c41034c0 2024-03-19T01:50:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:50 smithi092 ceph-mon[62574]: pgmap v287: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:50 smithi092 ceph-mon[62574]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:50:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:50 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2007629157' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-19T01:50:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:50 smithi092 ceph-mon[62574]: osdmap e83: 12 total, 12 up, 12 in 2024-03-19T01:50:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:50 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/660756190' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T01:50:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:50 smithi092 ceph-mon[62574]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-19T01:50:50.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.846+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2c402d440 con 0x7fe2c41034c0 2024-03-19T01:50:50.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:50 smithi144 ceph-mon[67199]: pgmap v287: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:50.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:50 smithi144 ceph-mon[67199]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:50:50.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:50 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2007629157' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-19T01:50:50.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:50 smithi144 ceph-mon[67199]: osdmap e83: 12 total, 12 up, 12 in 2024-03-19T01:50:50.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:50 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/660756190' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T01:50:50.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:50 smithi144 ceph-mon[67199]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-19T01:50:50.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.847+0000 7fe2c0ff9640 1 -- 172.21.15.92:0/3277605814 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22269 (secure 0 0 0) 0x7fe2bc097020 con 0x7fe2c41034c0 2024-03-19T01:50:50.848 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:50.848 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":84,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:50:49.881231+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe2ac0a2800 msgr2=0x7fe2ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe2ac0a2800 0x7fe2ac0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe2b402c7b0 tx=0x7fe2b4005fd0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c41034c0 msgr2=0x7fe2c419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c41034c0 0x7fe2c419f170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00eaa0 tx=0x7fe2bc00ef60 comp rx=0 tx=0).stop 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 shutdown_connections 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2c41034c0 0x7fe2c419f170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2c4102b00 0x7fe2c4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe2ac0a2800 0x7fe2ac0a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 --2- 172.21.15.92:0/3277605814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2c40ffb30 0x7fe2c4197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 >> 172.21.15.92:0/3277605814 conn(0x7fe2c40fb460 msgr2=0x7fe2c4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:50.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.849+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 shutdown_connections 2024-03-19T01:50:50.851 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.850+0000 7fe2c9fb3640 1 -- 172.21.15.92:0/3277605814 wait complete. 2024-03-19T01:50:50.857 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:50 smithi159 ceph-mon[67110]: pgmap v287: 33 pgs: 4 creating+peering, 1 active+clean, 28 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T01:50:50.857 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:50 smithi159 ceph-mon[67110]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-19T01:50:50.857 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:50 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2007629157' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-19T01:50:50.857 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:50 smithi159 ceph-mon[67110]: osdmap e83: 12 total, 12 up, 12 in 2024-03-19T01:50:50.857 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:50 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/660756190' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-19T01:50:50.857 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:50 smithi159 ceph-mon[67110]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-19T01:50:50.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:50:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.964+0000 7fa054ec6640 1 -- 172.21.15.92:0/349314384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa050104c90 msgr2=0x7fa050107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.964+0000 7fa054ec6640 1 --2- 172.21.15.92:0/349314384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa050104c90 0x7fa050107080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa038009900 tx=0x7fa03802bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.964+0000 7fa054ec6640 1 -- 172.21.15.92:0/349314384 shutdown_connections 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.964+0000 7fa054ec6640 1 --2- 172.21.15.92:0/349314384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0501075c0 0x7fa0501099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.964+0000 7fa054ec6640 1 --2- 172.21.15.92:0/349314384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa050104c90 0x7fa050107080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.964+0000 7fa054ec6640 1 --2- 172.21.15.92:0/349314384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050102360 0x7fa050104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.964+0000 7fa054ec6640 1 -- 172.21.15.92:0/349314384 >> 172.21.15.92:0/349314384 conn(0x7fa0500fbcb0 msgr2=0x7fa0500fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.964+0000 7fa054ec6640 1 -- 172.21.15.92:0/349314384 shutdown_connections 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.964+0000 7fa054ec6640 1 -- 172.21.15.92:0/349314384 wait complete. 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa054ec6640 1 Processor -- start 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa054ec6640 1 -- start start 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa054ec6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050102360 0x7fa050197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa054ec6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa050104c90 0x7fa0501980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa054ec6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0501075c0 0x7fa05019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa054ec6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010d490 con 0x7fa0501075c0 2024-03-19T01:50:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa054ec6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010d310 con 0x7fa050102360 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa054ec6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010d610 con 0x7fa050104c90 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0501075c0 0x7fa05019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04ed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0501075c0 0x7fa05019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35008/0 (socket says 172.21.15.92:35008) 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04ed9d640 1 -- 172.21.15.92:0/2487793061 learned_addr learned my addr 172.21.15.92:0/2487793061 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04e59c640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050102360 0x7fa050197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa047fff640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa050104c90 0x7fa0501980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04ed9d640 1 -- 172.21.15.92:0/2487793061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa050104c90 msgr2=0x7fa0501980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04ed9d640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa050104c90 0x7fa0501980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04ed9d640 1 -- 172.21.15.92:0/2487793061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050102360 msgr2=0x7fa050197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04ed9d640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050102360 0x7fa050197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04ed9d640 1 -- 172.21.15.92:0/2487793061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05019f780 con 0x7fa0501075c0 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa04ed9d640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0501075c0 0x7fa05019f110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa04800eaa0 tx=0x7fa04800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa0477fe640 1 -- 172.21.15.92:0/2487793061 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04800cdb0 con 0x7fa0501075c0 2024-03-19T01:50:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05019fa70 con 0x7fa0501075c0 2024-03-19T01:50:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa0477fe640 1 -- 172.21.15.92:0/2487793061 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa048005020 con 0x7fa0501075c0 2024-03-19T01:50:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.965+0000 7fa0477fe640 1 -- 172.21.15.92:0/2487793061 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0480116b0 con 0x7fa0501075c0 2024-03-19T01:50:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.966+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05019ff40 con 0x7fa0501075c0 2024-03-19T01:50:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.967+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0500fd7a0 con 0x7fa0501075c0 2024-03-19T01:50:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.968+0000 7fa0477fe640 1 -- 172.21.15.92:0/2487793061 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa048005560 con 0x7fa0501075c0 2024-03-19T01:50:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.969+0000 7fa0477fe640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa03c0a2800 0x7fa03c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.969+0000 7fa04e59c640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa03c0a2800 0x7fa03c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.969+0000 7fa04e59c640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa03c0a2800 0x7fa03c0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa040005d00 tx=0x7fa04000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.969+0000 7fa0477fe640 1 -- 172.21.15.92:0/2487793061 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fa0480cc7d0 con 0x7fa0501075c0 2024-03-19T01:50:50.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:50.974+0000 7fa0477fe640 1 -- 172.21.15.92:0/2487793061 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa048016030 con 0x7fa0501075c0 2024-03-19T01:50:51.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.151+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa050062420 con 0x7fa0501075c0 2024-03-19T01:50:51.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.153+0000 7fa0477fe640 1 -- 172.21.15.92:0/2487793061 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22269 (secure 0 0 0) 0x7fa04809b610 con 0x7fa0501075c0 2024-03-19T01:50:51.153 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:51.154 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":84,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:50:49.881231+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:50:51.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa03c0a2800 msgr2=0x7fa03c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:51.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa03c0a2800 0x7fa03c0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa040005d00 tx=0x7fa04000b410 comp rx=0 tx=0).stop 2024-03-19T01:50:51.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0501075c0 msgr2=0x7fa05019f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:51.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0501075c0 0x7fa05019f110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa04800eaa0 tx=0x7fa04800ef60 comp rx=0 tx=0).stop 2024-03-19T01:50:51.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 shutdown_connections 2024-03-19T01:50:51.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0501075c0 0x7fa05019f110 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa03c0a2800 0x7fa03c0a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa050104c90 0x7fa0501980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 --2- 172.21.15.92:0/2487793061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa050102360 0x7fa050197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 >> 172.21.15.92:0/2487793061 conn(0x7fa0500fbcb0 msgr2=0x7fa050102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:51.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 shutdown_connections 2024-03-19T01:50:51.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.155+0000 7fa054ec6640 1 -- 172.21.15.92:0/2487793061 wait complete. 2024-03-19T01:50:51.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-19T01:50:51.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.270+0000 7f7350b90640 1 -- 172.21.15.92:0/1561007975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c100720 msgr2=0x7f734c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:51.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.270+0000 7f7350b90640 1 --2- 172.21.15.92:0/1561007975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c100720 0x7f734c100b40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f733c009900 tx=0x7f733c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:51.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.270+0000 7f7350b90640 1 -- 172.21.15.92:0/1561007975 shutdown_connections 2024-03-19T01:50:51.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.270+0000 7f7350b90640 1 --2- 172.21.15.92:0/1561007975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734c1022d0 0x7f734c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.270+0000 7f7350b90640 1 --2- 172.21.15.92:0/1561007975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f734c101910 0x7f734c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.270+0000 7f7350b90640 1 --2- 172.21.15.92:0/1561007975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c100720 0x7f734c100b40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.270+0000 7f7350b90640 1 -- 172.21.15.92:0/1561007975 >> 172.21.15.92:0/1561007975 conn(0x7f734c0fbcb0 msgr2=0x7f734c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.270+0000 7f7350b90640 1 -- 172.21.15.92:0/1561007975 shutdown_connections 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.270+0000 7f7350b90640 1 -- 172.21.15.92:0/1561007975 wait complete. 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f7350b90640 1 Processor -- start 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f7350b90640 1 -- start start 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f7350b90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f734c100720 0x7f734c069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f7350b90640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734c101910 0x7f734c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f7350b90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c1022d0 0x7f734c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f7350b90640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f734c10c3f0 con 0x7f734c1022d0 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f7350b90640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f734c10c270 con 0x7f734c100720 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f7350b90640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f734c10c570 con 0x7f734c101910 2024-03-19T01:50:51.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f734ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c1022d0 0x7f734c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f734ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c1022d0 0x7f734c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35022/0 (socket says 172.21.15.92:35022) 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f734ad9d640 1 -- 172.21.15.92:0/259555110 learned_addr learned my addr 172.21.15.92:0/259555110 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.271+0000 7f7349d9b640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734c101910 0x7f734c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f734a59c640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f734c100720 0x7f734c069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f734ad9d640 1 -- 172.21.15.92:0/259555110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734c101910 msgr2=0x7f734c19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f734ad9d640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734c101910 0x7f734c19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f734ad9d640 1 -- 172.21.15.92:0/259555110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f734c100720 msgr2=0x7f734c069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f734ad9d640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f734c100720 0x7f734c069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f734ad9d640 1 -- 172.21.15.92:0/259555110 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c19f830 con 0x7f734c1022d0 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f734ad9d640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c1022d0 0x7f734c19ef70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f734400eaa0 tx=0x7f734400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f733b7fe640 1 -- 172.21.15.92:0/259555110 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344004d20 con 0x7f734c1022d0 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f733b7fe640 1 -- 172.21.15.92:0/259555110 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7344005020 con 0x7f734c1022d0 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f733b7fe640 1 -- 172.21.15.92:0/259555110 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73440116d0 con 0x7f734c1022d0 2024-03-19T01:50:51.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c19fb20 con 0x7f734c1022d0 2024-03-19T01:50:51.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.272+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c1a0020 con 0x7f734c1022d0 2024-03-19T01:50:51.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.273+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734c107b30 con 0x7f734c1022d0 2024-03-19T01:50:51.276 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.274+0000 7f733b7fe640 1 -- 172.21.15.92:0/259555110 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f73440051d0 con 0x7f734c1022d0 2024-03-19T01:50:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.275+0000 7f733b7fe640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f73340a2800 0x7f73340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.275+0000 7f734a59c640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f73340a2800 0x7f73340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.275+0000 7f733b7fe640 1 -- 172.21.15.92:0/259555110 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f73440ceb90 con 0x7f734c1022d0 2024-03-19T01:50:51.277 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.275+0000 7f734a59c640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f73340a2800 0x7f73340a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f733c02c8e0 tx=0x7f733c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:51.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.279+0000 7f733b7fe640 1 -- 172.21.15.92:0/259555110 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7344016030 con 0x7f734c1022d0 2024-03-19T01:50:51.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.489+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f734c02d3f0 con 0x7f734c1022d0 2024-03-19T01:50:51.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:51 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/660756190' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-19T01:50:51.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:51 smithi092 ceph-mon[62574]: osdmap e84: 12 total, 12 up, 12 in 2024-03-19T01:50:51.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:51 smithi092 ceph-mon[62574]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-19T01:50:51.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:51 smithi092 ceph-mon[62574]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-19T01:50:51.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:51 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3277605814' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:51 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] up:active 2024-03-19T01:50:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:51 smithi092 ceph-mon[62574]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-19T01:50:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:51 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2487793061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:51.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:51 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/259555110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T01:50:51.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:51 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/660756190' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-19T01:50:51.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:51 smithi144 ceph-mon[67199]: osdmap e84: 12 total, 12 up, 12 in 2024-03-19T01:50:51.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:51 smithi144 ceph-mon[67199]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-19T01:50:51.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:51 smithi144 ceph-mon[67199]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-19T01:50:51.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:51 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3277605814' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:51.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:51 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] up:active 2024-03-19T01:50:51.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:51 smithi144 ceph-mon[67199]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-19T01:50:51.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:51 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2487793061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:51.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:51 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/259555110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T01:50:51.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:51 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/660756190' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-19T01:50:51.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:51 smithi159 ceph-mon[67110]: osdmap e84: 12 total, 12 up, 12 in 2024-03-19T01:50:51.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:51 smithi159 ceph-mon[67110]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-19T01:50:51.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:51 smithi159 ceph-mon[67110]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-19T01:50:51.870 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:51 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3277605814' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:51.870 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:51 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] up:active 2024-03-19T01:50:51.870 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:51 smithi159 ceph-mon[67110]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-19T01:50:51.870 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:51 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2487793061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:51.870 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:51 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/259555110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-19T01:50:51.973 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.973+0000 7f733b7fe640 1 -- 172.21.15.92:0/259555110 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f734409bd80 con 0x7f734c1022d0 2024-03-19T01:50:51.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f73340a2800 msgr2=0x7f73340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:51.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f73340a2800 0x7f73340a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f733c02c8e0 tx=0x7f733c005fd0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c1022d0 msgr2=0x7f734c19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:51.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c1022d0 0x7f734c19ef70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f734400eaa0 tx=0x7f734400eea0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 shutdown_connections 2024-03-19T01:50:51.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f734c1022d0 0x7f734c19ef70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f73340a2800 0x7f73340a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734c101910 0x7f734c19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 --2- 172.21.15.92:0/259555110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f734c100720 0x7f734c069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:51.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 >> 172.21.15.92:0/259555110 conn(0x7f734c0fbcb0 msgr2=0x7f734c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:51.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 shutdown_connections 2024-03-19T01:50:51.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:51.975+0000 7f7350b90640 1 -- 172.21.15.92:0/259555110 wait complete. 2024-03-19T01:50:51.986 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-19T01:50:51.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-19T01:50:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.092+0000 7effc56c0640 1 -- 172.21.15.92:0/330110114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 msgr2=0x7effc0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.092+0000 7effc56c0640 1 --2- 172.21.15.92:0/330110114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 0x7effc0109c70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7effb8009900 tx=0x7effb802bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.092+0000 7effc56c0640 1 -- 172.21.15.92:0/330110114 shutdown_connections 2024-03-19T01:50:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.092+0000 7effc56c0640 1 --2- 172.21.15.92:0/330110114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 0x7effc0109c70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.092+0000 7effc56c0640 1 --2- 172.21.15.92:0/330110114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc0102a20 0x7effc0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.092+0000 7effc56c0640 1 --2- 172.21.15.92:0/330110114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc0101830 0x7effc0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.092+0000 7effc56c0640 1 -- 172.21.15.92:0/330110114 >> 172.21.15.92:0/330110114 conn(0x7effc00fcdc0 msgr2=0x7effc00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:52.095 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.092+0000 7effc56c0640 1 -- 172.21.15.92:0/330110114 shutdown_connections 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.092+0000 7effc56c0640 1 -- 172.21.15.92:0/330110114 wait complete. 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effc56c0640 1 Processor -- start 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effc56c0640 1 -- start start 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effc56c0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc0101830 0x7effc0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effc56c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc0102a20 0x7effc0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effc56c0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 0x7effc019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effc56c0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7effc010d490 con 0x7effc01033e0 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effc56c0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7effc010d310 con 0x7effc0101830 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effc56c0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7effc010d610 con 0x7effc0102a20 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 0x7effc019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 0x7effc019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35030/0 (socket says 172.21.15.92:35030) 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbf7fe640 1 -- 172.21.15.92:0/3301967659 learned_addr learned my addr 172.21.15.92:0/3301967659 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbe7fc640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc0102a20 0x7effc0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbeffd640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc0101830 0x7effc0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:52.096 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbf7fe640 1 -- 172.21.15.92:0/3301967659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc0102a20 msgr2=0x7effc0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbf7fe640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc0102a20 0x7effc0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbf7fe640 1 -- 172.21.15.92:0/3301967659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc0101830 msgr2=0x7effc0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbf7fe640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc0101830 0x7effc0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbf7fe640 1 -- 172.21.15.92:0/3301967659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc019f720 con 0x7effc01033e0 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.093+0000 7effbeffd640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc0101830 0x7effc0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.094+0000 7effbf7fe640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 0x7effc019f0b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effb802ce70 tx=0x7effb8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.094+0000 7effa7fff640 1 -- 172.21.15.92:0/3301967659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8041070 con 0x7effc01033e0 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.094+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc019f9b0 con 0x7effc01033e0 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.094+0000 7effa7fff640 1 -- 172.21.15.92:0/3301967659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb80054e0 con 0x7effc01033e0 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.094+0000 7effa7fff640 1 -- 172.21.15.92:0/3301967659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb80344a0 con 0x7effc01033e0 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.094+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc019fef0 con 0x7effc01033e0 2024-03-19T01:50:52.097 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.095+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc0108b50 con 0x7effc01033e0 2024-03-19T01:50:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.096+0000 7effa7fff640 1 -- 172.21.15.92:0/3301967659 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7effb8044070 con 0x7effc01033e0 2024-03-19T01:50:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.096+0000 7effa7fff640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7effa80a2800 0x7effa80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.097+0000 7effbeffd640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7effa80a2800 0x7effa80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.097+0000 7effa7fff640 1 -- 172.21.15.92:0/3301967659 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7effb80f06c0 con 0x7effc01033e0 2024-03-19T01:50:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.097+0000 7effbeffd640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7effa80a2800 0x7effa80a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7effc0102880 tx=0x7effb0009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.101+0000 7effa7fff640 1 -- 172.21.15.92:0/3301967659 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7effb80bd8b0 con 0x7effc01033e0 2024-03-19T01:50:52.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.315+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7effc002d3d0 con 0x7effc01033e0 2024-03-19T01:50:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:52 smithi092 ceph-mon[62574]: pgmap v290: 97 pgs: 20 active+clean, 2 creating+activating, 11 creating+peering, 64 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-03-19T01:50:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:52 smithi092 ceph-mon[62574]: osdmap e85: 12 total, 12 up, 12 in 2024-03-19T01:50:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:52 smithi092 ceph-mon[62574]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:50:52.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:52 smithi092 ceph-mon[62574]: Cluster is now healthy 2024-03-19T01:50:52.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:52 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/259555110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-19T01:50:52.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:52 smithi092 ceph-mon[62574]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-19T01:50:52.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:52 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3301967659' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-19T01:50:52.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:52 smithi144 ceph-mon[67199]: pgmap v290: 97 pgs: 20 active+clean, 2 creating+activating, 11 creating+peering, 64 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-03-19T01:50:52.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:52 smithi144 ceph-mon[67199]: osdmap e85: 12 total, 12 up, 12 in 2024-03-19T01:50:52.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:52 smithi144 ceph-mon[67199]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:50:52.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:52 smithi144 ceph-mon[67199]: Cluster is now healthy 2024-03-19T01:50:52.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:52 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/259555110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-19T01:50:52.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:52 smithi144 ceph-mon[67199]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-19T01:50:52.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:52 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3301967659' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-19T01:50:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:52 smithi159 ceph-mon[67110]: pgmap v290: 97 pgs: 20 active+clean, 2 creating+activating, 11 creating+peering, 64 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-03-19T01:50:52.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:52 smithi159 ceph-mon[67110]: osdmap e85: 12 total, 12 up, 12 in 2024-03-19T01:50:52.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:52 smithi159 ceph-mon[67110]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-19T01:50:52.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:52 smithi159 ceph-mon[67110]: Cluster is now healthy 2024-03-19T01:50:52.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:52 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/259555110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-19T01:50:52.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:52 smithi159 ceph-mon[67110]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-19T01:50:52.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:52 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3301967659' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-19T01:50:52.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.974+0000 7effa7fff640 1 -- 172.21.15.92:0/3301967659 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7effb80bd1b0 con 0x7effc01033e0 2024-03-19T01:50:52.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7effa80a2800 msgr2=0x7effa80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7effa80a2800 0x7effa80a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7effc0102880 tx=0x7effb0009540 comp rx=0 tx=0).stop 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 msgr2=0x7effc019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 0x7effc019f0b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effb802ce70 tx=0x7effb8005e40 comp rx=0 tx=0).stop 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 shutdown_connections 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effc01033e0 0x7effc019f0b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7effa80a2800 0x7effa80a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc0102a20 0x7effc0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 --2- 172.21.15.92:0/3301967659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc0101830 0x7effc0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 >> 172.21.15.92:0/3301967659 conn(0x7effc00fcdc0 msgr2=0x7effc0105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 shutdown_connections 2024-03-19T01:50:52.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:52.977+0000 7effc56c0640 1 -- 172.21.15.92:0/3301967659 wait complete. 2024-03-19T01:50:52.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-19T01:50:53.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.102+0000 7f6cdf92f640 1 -- 172.21.15.92:0/2266876431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd81033e0 msgr2=0x7f6cd8109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:53.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.102+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/2266876431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd81033e0 0x7f6cd8109c70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009ca0 tx=0x7f6cd002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 -- 172.21.15.92:0/2266876431 shutdown_connections 2024-03-19T01:50:53.105 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/2266876431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd81033e0 0x7f6cd8109c70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/2266876431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cd8102a20 0x7f6cd8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/2266876431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd8101830 0x7f6cd8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 -- 172.21.15.92:0/2266876431 >> 172.21.15.92:0/2266876431 conn(0x7f6cd80fcdc0 msgr2=0x7f6cd80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 -- 172.21.15.92:0/2266876431 shutdown_connections 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 -- 172.21.15.92:0/2266876431 wait complete. 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 Processor -- start 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 -- start start 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cd8101830 0x7f6cd80755d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd8102a20 0x7f6cd8073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd81033e0 0x7f6cd8074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd810d490 con 0x7f6cd8102a20 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd810d310 con 0x7f6cd81033e0 2024-03-19T01:50:53.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdf92f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd810d610 con 0x7f6cd8101830 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdceca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd8102a20 0x7f6cd8073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdceca640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd8102a20 0x7f6cd8073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35044/0 (socket says 172.21.15.92:35044) 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.103+0000 7f6cdceca640 1 -- 172.21.15.92:0/3244571554 learned_addr learned my addr 172.21.15.92:0/3244571554 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cdd6cb640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cd8101830 0x7f6cd80755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cddecc640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd81033e0 0x7f6cd8074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cdceca640 1 -- 172.21.15.92:0/3244571554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cd8101830 msgr2=0x7f6cd80755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cdceca640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cd8101830 0x7f6cd80755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cdceca640 1 -- 172.21.15.92:0/3244571554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd81033e0 msgr2=0x7f6cd8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cdceca640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd81033e0 0x7f6cd8074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cdceca640 1 -- 172.21.15.92:0/3244571554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd80749e0 con 0x7f6cd8102a20 2024-03-19T01:50:53.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cdceca640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd8102a20 0x7f6cd8073c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800d950 tx=0x7f6cc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:53.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cce7fc640 1 -- 172.21.15.92:0/3244571554 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8017030 con 0x7f6cd8102a20 2024-03-19T01:50:53.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd81a1bf0 con 0x7f6cd8102a20 2024-03-19T01:50:53.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cce7fc640 1 -- 172.21.15.92:0/3244571554 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc8004020 con 0x7f6cd8102a20 2024-03-19T01:50:53.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cce7fc640 1 -- 172.21.15.92:0/3244571554 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80047a0 con 0x7f6cd8102a20 2024-03-19T01:50:53.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.104+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd81a2140 con 0x7f6cd8102a20 2024-03-19T01:50:53.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.105+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd8108bd0 con 0x7f6cd8102a20 2024-03-19T01:50:53.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.106+0000 7f6cce7fc640 1 -- 172.21.15.92:0/3244571554 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f6cc80041d0 con 0x7f6cd8102a20 2024-03-19T01:50:53.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.107+0000 7f6cce7fc640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6cc40a2800 0x7f6cc40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:53.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.107+0000 7f6cdd6cb640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6cc40a2800 0x7f6cc40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:53.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.107+0000 7f6cce7fc640 1 -- 172.21.15.92:0/3244571554 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f6cc80ce390 con 0x7f6cd8102a20 2024-03-19T01:50:53.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.107+0000 7f6cdd6cb640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6cc40a2800 0x7f6cc40a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8102880 tx=0x7f6cd400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:53.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.111+0000 7f6cce7fc640 1 -- 172.21.15.92:0/3244571554 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6cc809b580 con 0x7f6cd8102a20 2024-03-19T01:50:53.324 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.324+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6cd802d1f0 con 0x7f6cd8102a20 2024-03-19T01:50:53.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:53 smithi092 ceph-mon[62574]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-19T01:50:53.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:53 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3301967659' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-19T01:50:53.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:53 smithi092 ceph-mon[62574]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-19T01:50:53.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:53 smithi092 ceph-mon[62574]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-19T01:50:53.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:53 smithi092 ceph-mon[62574]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-19T01:50:53.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:53 smithi092 ceph-mon[62574]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-19T01:50:53.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:53 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3244571554' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T01:50:53.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:53 smithi144 ceph-mon[67199]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-19T01:50:53.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:53 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3301967659' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-19T01:50:53.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:53 smithi144 ceph-mon[67199]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-19T01:50:53.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:53 smithi144 ceph-mon[67199]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-19T01:50:53.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:53 smithi144 ceph-mon[67199]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-19T01:50:53.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:53 smithi144 ceph-mon[67199]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-19T01:50:53.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:53 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3244571554' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T01:50:53.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:53 smithi159 ceph-mon[67110]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-19T01:50:53.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:53 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3301967659' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-19T01:50:53.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:53 smithi159 ceph-mon[67110]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-19T01:50:53.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:53 smithi159 ceph-mon[67110]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-19T01:50:53.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:53 smithi159 ceph-mon[67110]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-19T01:50:53.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:53 smithi159 ceph-mon[67110]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-19T01:50:53.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:53 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3244571554' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-19T01:50:53.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.981+0000 7f6cce7fc640 1 -- 172.21.15.92:0/3244571554 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f6cc809ae80 con 0x7f6cd8102a20 2024-03-19T01:50:53.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6cc40a2800 msgr2=0x7f6cc40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:53.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6cc40a2800 0x7f6cc40a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8102880 tx=0x7f6cd400b390 comp rx=0 tx=0).stop 2024-03-19T01:50:53.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd8102a20 msgr2=0x7f6cd8073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:53.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd8102a20 0x7f6cd8073c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800d950 tx=0x7f6cc800de10 comp rx=0 tx=0).stop 2024-03-19T01:50:53.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 shutdown_connections 2024-03-19T01:50:53.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd81033e0 0x7f6cd8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cd8102a20 0x7f6cd8073c10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6cc40a2800 0x7f6cc40a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 --2- 172.21.15.92:0/3244571554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cd8101830 0x7f6cd80755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:53.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 >> 172.21.15.92:0/3244571554 conn(0x7f6cd80fcdc0 msgr2=0x7f6cd8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:53.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 shutdown_connections 2024-03-19T01:50:53.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:53.984+0000 7f6cdf92f640 1 -- 172.21.15.92:0/3244571554 wait complete. 2024-03-19T01:50:54.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-19T01:50:54.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.110+0000 7f40c877e640 1 -- 172.21.15.92:0/3739452303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 msgr2=0x7f40c0102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:54.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.110+0000 7f40c877e640 1 --2- 172.21.15.92:0/3739452303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 0x7f40c0102ec0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009900 tx=0x7f40b002bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.110+0000 7f40c877e640 1 -- 172.21.15.92:0/3739452303 shutdown_connections 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.110+0000 7f40c877e640 1 --2- 172.21.15.92:0/3739452303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c0103400 0x7f40c0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.110+0000 7f40c877e640 1 --2- 172.21.15.92:0/3739452303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 0x7f40c0102ec0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.110+0000 7f40c877e640 1 --2- 172.21.15.92:0/3739452303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0101850 0x7f40c0101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.110+0000 7f40c877e640 1 -- 172.21.15.92:0/3739452303 >> 172.21.15.92:0/3739452303 conn(0x7f40c00fcdc0 msgr2=0x7f40c00ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.110+0000 7f40c877e640 1 -- 172.21.15.92:0/3739452303 shutdown_connections 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.110+0000 7f40c877e640 1 -- 172.21.15.92:0/3739452303 wait complete. 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c877e640 1 Processor -- start 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c877e640 1 -- start start 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c877e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0101850 0x7f40c0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c877e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 0x7f40c0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c877e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c0103400 0x7f40c019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c877e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f40c010d4b0 con 0x7f40c0102a40 2024-03-19T01:50:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c877e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f40c010d330 con 0x7f40c0101850 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c877e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f40c010d630 con 0x7f40c0103400 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c5d19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 0x7f40c0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c5d19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 0x7f40c0198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37774/0 (socket says 172.21.15.92:37774) 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c5d19640 1 -- 172.21.15.92:0/2044483378 learned_addr learned my addr 172.21.15.92:0/2044483378 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c6d1b640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c0103400 0x7f40c019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c651a640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0101850 0x7f40c0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c5d19640 1 -- 172.21.15.92:0/2044483378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c0103400 msgr2=0x7f40c019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c5d19640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c0103400 0x7f40c019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c5d19640 1 -- 172.21.15.92:0/2044483378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0101850 msgr2=0x7f40c0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c5d19640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0101850 0x7f40c0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.111+0000 7f40c5d19640 1 -- 172.21.15.92:0/2044483378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c019f770 con 0x7f40c0102a40 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.112+0000 7f40c5d19640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 0x7f40c0198090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009230 tx=0x7f40b002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:54.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.112+0000 7f40b77fe640 1 -- 172.21.15.92:0/2044483378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0041070 con 0x7f40c0102a40 2024-03-19T01:50:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.112+0000 7f40b77fe640 1 -- 172.21.15.92:0/2044483378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0004450 con 0x7f40c0102a40 2024-03-19T01:50:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.112+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c019fa00 con 0x7f40c0102a40 2024-03-19T01:50:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.112+0000 7f40b77fe640 1 -- 172.21.15.92:0/2044483378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0003e20 con 0x7f40c0102a40 2024-03-19T01:50:54.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.112+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c019fef0 con 0x7f40c0102a40 2024-03-19T01:50:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.113+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c0108bf0 con 0x7f40c0102a40 2024-03-19T01:50:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.114+0000 7f40b77fe640 1 -- 172.21.15.92:0/2044483378 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f40b0044070 con 0x7f40c0102a40 2024-03-19T01:50:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.114+0000 7f40b77fe640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f40ac0a2800 0x7f40ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.114+0000 7f40c651a640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f40ac0a2800 0x7f40ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.114+0000 7f40b77fe640 1 -- 172.21.15.92:0/2044483378 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f40b00f0810 con 0x7f40c0102a40 2024-03-19T01:50:54.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.115+0000 7f40c651a640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f40ac0a2800 0x7f40ac0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f40c01028a0 tx=0x7f40bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:54.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.118+0000 7f40b77fe640 1 -- 172.21.15.92:0/2044483378 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f40b00bd3b0 con 0x7f40c0102a40 2024-03-19T01:50:54.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.330+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f40c002d1f0 con 0x7f40c0102a40 2024-03-19T01:50:54.567 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:54 smithi092 ceph-mon[62574]: pgmap v292: 97 pgs: 57 active+clean, 2 creating+activating, 11 creating+peering, 27 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s wr, 9 op/s 2024-03-19T01:50:54.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:54 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3244571554' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-19T01:50:54.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:54 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] up:active 2024-03-19T01:50:54.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:54 smithi092 ceph-mon[62574]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-19T01:50:54.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:54 smithi092 ceph-mon[62574]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-19T01:50:54.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:54 smithi092 ceph-mon[62574]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-19T01:50:54.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:54 smithi092 ceph-mon[62574]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-19T01:50:54.568 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:54 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2044483378' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T01:50:54.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:54 smithi144 ceph-mon[67199]: pgmap v292: 97 pgs: 57 active+clean, 2 creating+activating, 11 creating+peering, 27 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s wr, 9 op/s 2024-03-19T01:50:54.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:54 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3244571554' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-19T01:50:54.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:54 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] up:active 2024-03-19T01:50:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:54 smithi144 ceph-mon[67199]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-19T01:50:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:54 smithi144 ceph-mon[67199]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-19T01:50:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:54 smithi144 ceph-mon[67199]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-19T01:50:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:54 smithi144 ceph-mon[67199]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-19T01:50:54.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:54 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2044483378' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T01:50:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:54 smithi159 ceph-mon[67110]: pgmap v292: 97 pgs: 57 active+clean, 2 creating+activating, 11 creating+peering, 27 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s wr, 9 op/s 2024-03-19T01:50:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:54 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3244571554' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-19T01:50:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:54 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] up:active 2024-03-19T01:50:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:54 smithi159 ceph-mon[67110]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-19T01:50:54.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:54 smithi159 ceph-mon[67110]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-19T01:50:54.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:54 smithi159 ceph-mon[67110]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-19T01:50:54.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:54 smithi159 ceph-mon[67110]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-19T01:50:54.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:54 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2044483378' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-19T01:50:54.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.988+0000 7f40b77fe640 1 -- 172.21.15.92:0/2044483378 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f40b00c1450 con 0x7f40c0102a40 2024-03-19T01:50:54.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f40ac0a2800 msgr2=0x7f40ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f40ac0a2800 0x7f40ac0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f40c01028a0 tx=0x7f40bc005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 msgr2=0x7f40c0198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 0x7f40c0198090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009230 tx=0x7f40b002c650 comp rx=0 tx=0).stop 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 shutdown_connections 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f40ac0a2800 0x7f40ac0a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40c0103400 0x7f40c019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0102a40 0x7f40c0198090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 --2- 172.21.15.92:0/2044483378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0101850 0x7f40c0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 >> 172.21.15.92:0/2044483378 conn(0x7f40c00fcdc0 msgr2=0x7f40c0105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 shutdown_connections 2024-03-19T01:50:54.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:54.990+0000 7f40c877e640 1 -- 172.21.15.92:0/2044483378 wait complete. 2024-03-19T01:50:55.006 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-19T01:50:55.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-19T01:50:55.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.113+0000 7f063e88c640 1 -- 172.21.15.92:0/695977181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638074200 msgr2=0x7f0638074660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:55.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.113+0000 7f063e88c640 1 --2- 172.21.15.92:0/695977181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638074200 0x7f0638074660 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0628009900 tx=0x7f062802bcd0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.113+0000 7f063e88c640 1 -- 172.21.15.92:0/695977181 shutdown_connections 2024-03-19T01:50:55.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.113+0000 7f063e88c640 1 --2- 172.21.15.92:0/695977181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0638109700 0x7f063810bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.113+0000 7f063e88c640 1 --2- 172.21.15.92:0/695977181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638074200 0x7f0638074660 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.113+0000 7f063e88c640 1 --2- 172.21.15.92:0/695977181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06380757e0 0x7f0638073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.113+0000 7f063e88c640 1 -- 172.21.15.92:0/695977181 >> 172.21.15.92:0/695977181 conn(0x7f06380fcdc0 msgr2=0x7f06380ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:55.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.113+0000 7f063e88c640 1 -- 172.21.15.92:0/695977181 shutdown_connections 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.113+0000 7f063e88c640 1 -- 172.21.15.92:0/695977181 wait complete. 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063e88c640 1 Processor -- start 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063e88c640 1 -- start start 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063e88c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0638074200 0x7f0638199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063e88c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06380757e0 0x7f063819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063e88c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638109700 0x7f06381a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063e88c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f063810f6e0 con 0x7f0638109700 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063e88c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f063810f560 con 0x7f06380757e0 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063e88c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f063810f860 con 0x7f0638074200 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063ce29640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638109700 0x7f06381a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f0637fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0638074200 0x7f0638199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063ce29640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638109700 0x7f06381a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37792/0 (socket says 172.21.15.92:37792) 2024-03-19T01:50:55.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.114+0000 7f063ce29640 1 -- 172.21.15.92:0/573435630 learned_addr learned my addr 172.21.15.92:0/573435630 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f06377fe640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06380757e0 0x7f063819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f063ce29640 1 -- 172.21.15.92:0/573435630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0638074200 msgr2=0x7f0638199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f063ce29640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0638074200 0x7f0638199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f063ce29640 1 -- 172.21.15.92:0/573435630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06380757e0 msgr2=0x7f063819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f063ce29640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06380757e0 0x7f063819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f063ce29640 1 -- 172.21.15.92:0/573435630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06381a19d0 con 0x7f0638109700 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f063ce29640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638109700 0x7f06381a1360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f063000eb10 tx=0x7f063000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f06357fa640 1 -- 172.21.15.92:0/573435630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630004b20 con 0x7f0638109700 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f06357fa640 1 -- 172.21.15.92:0/573435630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0630004cc0 con 0x7f0638109700 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f06357fa640 1 -- 172.21.15.92:0/573435630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630010760 con 0x7f0638109700 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06381a1cc0 con 0x7f0638109700 2024-03-19T01:50:55.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.115+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381a2190 con 0x7f0638109700 2024-03-19T01:50:55.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.116+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06381084c0 con 0x7f0638109700 2024-03-19T01:50:55.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.117+0000 7f06357fa640 1 -- 172.21.15.92:0/573435630 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0630005020 con 0x7f0638109700 2024-03-19T01:50:55.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.118+0000 7f06357fa640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f06200a2880 0x7f06200a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:55.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.118+0000 7f0637fff640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f06200a2880 0x7f06200a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:55.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.118+0000 7f06357fa640 1 -- 172.21.15.92:0/573435630 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f06300cedf0 con 0x7f0638109700 2024-03-19T01:50:55.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.118+0000 7f0637fff640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f06200a2880 0x7f06200a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f062c009bd0 tx=0x7f062c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:55.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.122+0000 7f06357fa640 1 -- 172.21.15.92:0/573435630 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f063009bfe0 con 0x7f0638109700 2024-03-19T01:50:55.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.331+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0638061c60 con 0x7f06200a2880 2024-03-19T01:50:55.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:50:55.336+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:50:55.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:50:55.336+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:50:55.811 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:50:55.336+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:50:55.811 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:50:55.336+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:50:55.811 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:50:55.336+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:50:55.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.979+0000 7f06357fa640 1 -- 172.21.15.92:0/573435630 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0638061c60 con 0x7f06200a2880 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f06200a2880 msgr2=0x7f06200a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f06200a2880 0x7f06200a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f062c009bd0 tx=0x7f062c005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638109700 msgr2=0x7f06381a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638109700 0x7f06381a1360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f063000eb10 tx=0x7f063000c2d0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 shutdown_connections 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0638109700 0x7f06381a1360 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06380757e0 0x7f063819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f06200a2880 0x7f06200a4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 --2- 172.21.15.92:0/573435630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0638074200 0x7f0638199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 >> 172.21.15.92:0/573435630 conn(0x7f06380fcdc0 msgr2=0x7f0638105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 shutdown_connections 2024-03-19T01:50:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:55.981+0000 7f063e88c640 1 -- 172.21.15.92:0/573435630 wait complete. 2024-03-19T01:50:55.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-19T01:50:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.103+0000 7f78ff84b640 1 -- 172.21.15.92:0/2097803186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80ffb20 msgr2=0x7f78f80fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.103+0000 7f78ff84b640 1 --2- 172.21.15.92:0/2097803186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80ffb20 0x7f78f80fff40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009930 tx=0x7f78f402bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.103+0000 7f78ff84b640 1 -- 172.21.15.92:0/2097803186 shutdown_connections 2024-03-19T01:50:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.103+0000 7f78ff84b640 1 --2- 172.21.15.92:0/2097803186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78f8103550 0x7f78f8109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.103+0000 7f78ff84b640 1 --2- 172.21.15.92:0/2097803186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f8102b90 0x7f78f8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.103+0000 7f78ff84b640 1 --2- 172.21.15.92:0/2097803186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80ffb20 0x7f78f80fff40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.103+0000 7f78ff84b640 1 -- 172.21.15.92:0/2097803186 >> 172.21.15.92:0/2097803186 conn(0x7f78f80fb450 msgr2=0x7f78f80fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 -- 172.21.15.92:0/2097803186 shutdown_connections 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 -- 172.21.15.92:0/2097803186 wait complete. 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 Processor -- start 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 -- start start 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f80ffb20 0x7f78f8102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78f8102b90 0x7f78f8100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8103550 0x7f78f8101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78f810d4d0 con 0x7f78f8103550 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78f810d350 con 0x7f78f8102b90 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78ff84b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78f810d650 con 0x7f78f80ffb20 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78fdde8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8103550 0x7f78f8101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78fd5e7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f80ffb20 0x7f78f8102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78fdde8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8103550 0x7f78f8101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37830/0 (socket says 172.21.15.92:37830) 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78fdde8640 1 -- 172.21.15.92:0/1510204990 learned_addr learned my addr 172.21.15.92:0/1510204990 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:56.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.104+0000 7f78fcde6640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78f8102b90 0x7f78f8100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78fdde8640 1 -- 172.21.15.92:0/1510204990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f80ffb20 msgr2=0x7f78f8102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78fdde8640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f80ffb20 0x7f78f8102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78fdde8640 1 -- 172.21.15.92:0/1510204990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78f8102b90 msgr2=0x7f78f8100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78fdde8640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78f8102b90 0x7f78f8100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78fdde8640 1 -- 172.21.15.92:0/1510204990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f8101910 con 0x7f78f8103550 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78fcde6640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78f8102b90 0x7f78f8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78fdde8640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8103550 0x7f78f8101080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f78f000eb10 tx=0x7f78f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78ee7fc640 1 -- 172.21.15.92:0/1510204990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0004b20 con 0x7f78f8103550 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78ee7fc640 1 -- 172.21.15.92:0/1510204990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f0004cc0 con 0x7f78f8103550 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f81a1ce0 con 0x7f78f8103550 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78ee7fc640 1 -- 172.21.15.92:0/1510204990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0010760 con 0x7f78f8103550 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.105+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f81a2180 con 0x7f78f8103550 2024-03-19T01:50:56.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.106+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f8108db0 con 0x7f78f8103550 2024-03-19T01:50:56.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.107+0000 7f78ee7fc640 1 -- 172.21.15.92:0/1510204990 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f78f0005020 con 0x7f78f8103550 2024-03-19T01:50:56.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.107+0000 7f78ee7fc640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78e40a2800 0x7f78e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.108+0000 7f78fd5e7640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78e40a2800 0x7f78e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.108+0000 7f78ee7fc640 1 -- 172.21.15.92:0/1510204990 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f78f00cefa0 con 0x7f78f8103550 2024-03-19T01:50:56.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.108+0000 7f78fd5e7640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78e40a2800 0x7f78e40a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f78f402c7b0 tx=0x7f78f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:56.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.112+0000 7f78ee7fc640 1 -- 172.21.15.92:0/1510204990 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f78f009c190 con 0x7f78f8103550 2024-03-19T01:50:56.205 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: 2024-03-19T01:50:55.862+0000 7f2ab486a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-19T01:50:56.205 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: 2024-03-19T01:50:55.862+0000 7f2ab486a700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-19T01:50:56.205 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: 2024-03-19T01:50:55.862+0000 7f2ab486a700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2044483378' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] up:active 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: pgmap v293: 97 pgs: 88 active+clean, 2 creating+activating, 7 creating+peering; 580 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 11 op/s 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] up:active 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-19T01:50:56.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: Cluster is now healthy 2024-03-19T01:50:56.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-03-19T01:50:56.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-19T01:50:56.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:55 smithi159 ceph-mon[67110]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-19T01:50:56.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2044483378' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-19T01:50:56.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] up:active 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: pgmap v293: 97 pgs: 88 active+clean, 2 creating+activating, 7 creating+peering; 580 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 11 op/s 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] up:active 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: Cluster is now healthy 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-19T01:50:56.311 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:55 smithi092 ceph-mon[62574]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-19T01:50:56.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.314+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f78f8061cf0 con 0x7f78e40a2800 2024-03-19T01:50:56.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.335+0000 7f78ee7fc640 1 -- 172.21.15.92:0/1510204990 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f78f8061cf0 con 0x7f78e40a2800 2024-03-19T01:50:56.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.336+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78e40a2800 msgr2=0x7f78e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.336+0000 7f78ff84b640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78e40a2800 0x7f78e40a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f78f402c7b0 tx=0x7f78f4005fd0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.336+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8103550 msgr2=0x7f78f8101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.336+0000 7f78ff84b640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8103550 0x7f78f8101080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f78f000eb10 tx=0x7f78f000c2d0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.336+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 shutdown_connections 2024-03-19T01:50:56.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.336+0000 7f78ff84b640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8103550 0x7f78f8101080 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.337+0000 7f78ff84b640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78f8102b90 0x7f78f8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.337+0000 7f78ff84b640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f78e40a2800 0x7f78e40a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.337+0000 7f78ff84b640 1 --2- 172.21.15.92:0/1510204990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78f80ffb20 0x7f78f8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.337+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 >> 172.21.15.92:0/1510204990 conn(0x7f78f80fb450 msgr2=0x7f78f8105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:56.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.337+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 shutdown_connections 2024-03-19T01:50:56.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.337+0000 7f78ff84b640 1 -- 172.21.15.92:0/1510204990 wait complete. 2024-03-19T01:50:56.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2044483378' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-19T01:50:56.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] up:active 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: pgmap v293: 97 pgs: 88 active+clean, 2 creating+activating, 7 creating+peering; 580 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s wr, 11 op/s 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] up:active 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: Cluster is now healthy 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-03-19T01:50:56.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-19T01:50:56.348 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:55 smithi144 ceph-mon[67199]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-19T01:50:56.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-19T01:50:56.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.454+0000 7fd64bc27640 1 -- 172.21.15.92:0/2451134362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 msgr2=0x7fd6440fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.454+0000 7fd64bc27640 1 --2- 172.21.15.92:0/2451134362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 0x7fd6440fff20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd640009cd0 tx=0x7fd64002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.454+0000 7fd64bc27640 1 -- 172.21.15.92:0/2451134362 shutdown_connections 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.454+0000 7fd64bc27640 1 --2- 172.21.15.92:0/2451134362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd644103490 0x7fd644109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.454+0000 7fd64bc27640 1 --2- 172.21.15.92:0/2451134362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd644102ad0 0x7fd644102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.454+0000 7fd64bc27640 1 --2- 172.21.15.92:0/2451134362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 0x7fd6440fff20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.454+0000 7fd64bc27640 1 -- 172.21.15.92:0/2451134362 >> 172.21.15.92:0/2451134362 conn(0x7fd6440fb430 msgr2=0x7fd6440fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.454+0000 7fd64bc27640 1 -- 172.21.15.92:0/2451134362 shutdown_connections 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.454+0000 7fd64bc27640 1 -- 172.21.15.92:0/2451134362 wait complete. 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd64bc27640 1 Processor -- start 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd64bc27640 1 -- start start 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd64bc27640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 0x7fd644197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd64bc27640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd644102ad0 0x7fd644198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd64bc27640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd644103490 0x7fd64419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd64bc27640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd64410d4b0 con 0x7fd6440ffb00 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd64bc27640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd64410d330 con 0x7fd644103490 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd64bc27640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd64410d630 con 0x7fd644102ad0 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd6499c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 0x7fd644197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd6499c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 0x7fd644197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37842/0 (socket says 172.21.15.92:37842) 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd6499c3640 1 -- 172.21.15.92:0/3932292889 learned_addr learned my addr 172.21.15.92:0/3932292889 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd6491c2640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd644102ad0 0x7fd644198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd64a1c4640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd644103490 0x7fd64419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd6499c3640 1 -- 172.21.15.92:0/3932292889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd644102ad0 msgr2=0x7fd644198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd6499c3640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd644102ad0 0x7fd644198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd6499c3640 1 -- 172.21.15.92:0/3932292889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd644103490 msgr2=0x7fd64419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd6499c3640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd644103490 0x7fd64419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.455+0000 7fd6499c3640 1 -- 172.21.15.92:0/3932292889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64419f7f0 con 0x7fd6440ffb00 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.456+0000 7fd6499c3640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 0x7fd644197bd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd64002c700 tx=0x7fd640034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.456+0000 7fd63affd640 1 -- 172.21.15.92:0/3932292889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640039040 con 0x7fd6440ffb00 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.456+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64419fa80 con 0x7fd6440ffb00 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.456+0000 7fd63affd640 1 -- 172.21.15.92:0/3932292889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd640035b60 con 0x7fd6440ffb00 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.456+0000 7fd63affd640 1 -- 172.21.15.92:0/3932292889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6400042a0 con 0x7fd6440ffb00 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.456+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64419ffc0 con 0x7fd6440ffb00 2024-03-19T01:50:56.460 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.457+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd644108cd0 con 0x7fd6440ffb00 2024-03-19T01:50:56.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.458+0000 7fd63affd640 1 -- 172.21.15.92:0/3932292889 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd640046030 con 0x7fd6440ffb00 2024-03-19T01:50:56.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.458+0000 7fd63affd640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd6300a2800 0x7fd6300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.459+0000 7fd6491c2640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd6300a2800 0x7fd6300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.459+0000 7fd63affd640 1 -- 172.21.15.92:0/3932292889 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd6400f0800 con 0x7fd6440ffb00 2024-03-19T01:50:56.461 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.459+0000 7fd6491c2640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd6300a2800 0x7fd6300a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd6441990e0 tx=0x7fd63400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:56.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.462+0000 7fd63affd640 1 -- 172.21.15.92:0/3932292889 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd64004d060 con 0x7fd6440ffb00 2024-03-19T01:50:56.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.665+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd644061c60 con 0x7fd6300a2800 2024-03-19T01:50:56.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.669+0000 7fd63affd640 1 -- 172.21.15.92:0/3932292889 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd644061c60 con 0x7fd6300a2800 2024-03-19T01:50:56.669 INFO:teuthology.orchestra.run.smithi092.stdout:/volumes/qa/sv_0/58dd6d62-8573-4eb8-a7f0-93bb8fcf9a01 2024-03-19T01:50:56.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd6300a2800 msgr2=0x7fd6300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd6300a2800 0x7fd6300a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd6441990e0 tx=0x7fd63400b390 comp rx=0 tx=0).stop 2024-03-19T01:50:56.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 msgr2=0x7fd644197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 0x7fd644197bd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd64002c700 tx=0x7fd640034760 comp rx=0 tx=0).stop 2024-03-19T01:50:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 shutdown_connections 2024-03-19T01:50:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd644103490 0x7fd64419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd6300a2800 0x7fd6300a4cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd644102ad0 0x7fd644198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 --2- 172.21.15.92:0/3932292889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6440ffb00 0x7fd644197bd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.670+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 >> 172.21.15.92:0/3932292889 conn(0x7fd6440fb430 msgr2=0x7fd644105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.671+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 shutdown_connections 2024-03-19T01:50:56.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.671+0000 7fd64bc27640 1 -- 172.21.15.92:0/3932292889 wait complete. 2024-03-19T01:50:56.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-19T01:50:56.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.787+0000 7f4249716640 1 -- 172.21.15.92:0/3012916424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244104c50 msgr2=0x7f4244107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.787+0000 7f4249716640 1 --2- 172.21.15.92:0/3012916424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244104c50 0x7f4244107040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4238009900 tx=0x7f423802bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:56.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.787+0000 7f4249716640 1 -- 172.21.15.92:0/3012916424 shutdown_connections 2024-03-19T01:50:56.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.787+0000 7f4249716640 1 --2- 172.21.15.92:0/3012916424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244107580 0x7f4244109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.787+0000 7f4249716640 1 --2- 172.21.15.92:0/3012916424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244104c50 0x7f4244107040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.787+0000 7f4249716640 1 --2- 172.21.15.92:0/3012916424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4244102320 0x7f4244104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.787+0000 7f4249716640 1 -- 172.21.15.92:0/3012916424 >> 172.21.15.92:0/3012916424 conn(0x7f42440fbcb0 msgr2=0x7f42440fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.787+0000 7f4249716640 1 -- 172.21.15.92:0/3012916424 shutdown_connections 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.787+0000 7f4249716640 1 -- 172.21.15.92:0/3012916424 wait complete. 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f4249716640 1 Processor -- start 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f4249716640 1 -- start start 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f4249716640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4244102320 0x7f4244197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f4249716640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244104c50 0x7f42441980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f4249716640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244107580 0x7f424419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f4249716640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f424410d4a0 con 0x7f4244107580 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f4249716640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f424410d320 con 0x7f4244102320 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f4249716640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f424410d620 con 0x7f4244104c50 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f42437fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244107580 0x7f424419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f42437fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244107580 0x7f424419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37856/0 (socket says 172.21.15.92:37856) 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f42437fe640 1 -- 172.21.15.92:0/30043490 learned_addr learned my addr 172.21.15.92:0/30043490 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f42427fc640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244104c50 0x7f42441980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.788+0000 7f4242ffd640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4244102320 0x7f4244197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f42437fe640 1 -- 172.21.15.92:0/30043490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244104c50 msgr2=0x7f42441980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f42437fe640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244104c50 0x7f42441980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f42437fe640 1 -- 172.21.15.92:0/30043490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4244102320 msgr2=0x7f4244197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f42437fe640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4244102320 0x7f4244197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f42437fe640 1 -- 172.21.15.92:0/30043490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424419f7c0 con 0x7f4244107580 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f42437fe640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244107580 0x7f424419f150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f423c00e970 tx=0x7f423c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f422bfff640 1 -- 172.21.15.92:0/30043490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c00ccc0 con 0x7f4244107580 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424419fab0 con 0x7f4244107580 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f422bfff640 1 -- 172.21.15.92:0/30043490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f423c00ce60 con 0x7f4244107580 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f422bfff640 1 -- 172.21.15.92:0/30043490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c010450 con 0x7f4244107580 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.789+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42441a0020 con 0x7f4244107580 2024-03-19T01:50:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.790+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42440fd7a0 con 0x7f4244107580 2024-03-19T01:50:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.791+0000 7f422bfff640 1 -- 172.21.15.92:0/30043490 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f423c005020 con 0x7f4244107580 2024-03-19T01:50:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.791+0000 7f422bfff640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f422c0a2800 0x7f422c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.791+0000 7f4242ffd640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f422c0a2800 0x7f422c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.792+0000 7f422bfff640 1 -- 172.21.15.92:0/30043490 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f423c0ced30 con 0x7f4244107580 2024-03-19T01:50:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.792+0000 7f4242ffd640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f422c0a2800 0x7f422c0a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4234005fd0 tx=0x7f4234009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:56.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:56.796+0000 7f422bfff640 1 -- 172.21.15.92:0/30043490 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f423c09bf20 con 0x7f4244107580 2024-03-19T01:50:57.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.002+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4244061c60 con 0x7f422c0a2800 2024-03-19T01:50:57.026 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.026+0000 7f422bfff640 1 -- 172.21.15.92:0/30043490 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4244061c60 con 0x7f422c0a2800 2024-03-19T01:50:57.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f422c0a2800 msgr2=0x7f422c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f422c0a2800 0x7f422c0a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4234005fd0 tx=0x7f4234009500 comp rx=0 tx=0).stop 2024-03-19T01:50:57.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244107580 msgr2=0x7f424419f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244107580 0x7f424419f150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f423c00e970 tx=0x7f423c00ed70 comp rx=0 tx=0).stop 2024-03-19T01:50:57.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 shutdown_connections 2024-03-19T01:50:57.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4244107580 0x7f424419f150 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.028 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f422c0a2800 0x7f422c0a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4244104c50 0x7f42441980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 --2- 172.21.15.92:0/30043490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4244102320 0x7f4244197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 >> 172.21.15.92:0/30043490 conn(0x7f42440fbcb0 msgr2=0x7f4244102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:57.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 shutdown_connections 2024-03-19T01:50:57.029 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.028+0000 7f4249716640 1 -- 172.21.15.92:0/30043490 wait complete. 2024-03-19T01:50:57.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-19T01:50:57.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.146+0000 7fbc568bf640 1 -- 172.21.15.92:0/4049030116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc50100720 msgr2=0x7fbc50100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.146+0000 7fbc568bf640 1 --2- 172.21.15.92:0/4049030116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc50100720 0x7fbc50100b40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009900 tx=0x7fbc4402bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:57.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.146+0000 7fbc568bf640 1 -- 172.21.15.92:0/4049030116 shutdown_connections 2024-03-19T01:50:57.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.146+0000 7fbc568bf640 1 --2- 172.21.15.92:0/4049030116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc501022d0 0x7fbc50108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.146+0000 7fbc568bf640 1 --2- 172.21.15.92:0/4049030116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc50101910 0x7fbc50101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.146+0000 7fbc568bf640 1 --2- 172.21.15.92:0/4049030116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc50100720 0x7fbc50100b40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.146+0000 7fbc568bf640 1 -- 172.21.15.92:0/4049030116 >> 172.21.15.92:0/4049030116 conn(0x7fbc500fbcb0 msgr2=0x7fbc500fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:57.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.146+0000 7fbc568bf640 1 -- 172.21.15.92:0/4049030116 shutdown_connections 2024-03-19T01:50:57.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.146+0000 7fbc568bf640 1 -- 172.21.15.92:0/4049030116 wait complete. 2024-03-19T01:50:57.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc568bf640 1 Processor -- start 2024-03-19T01:50:57.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc568bf640 1 -- start start 2024-03-19T01:50:57.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc568bf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc50100720 0x7fbc50196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc568bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc50101910 0x7fbc50196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc568bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc501022d0 0x7fbc5019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc568bf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5010c370 con 0x7fbc501022d0 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc568bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5010c1f0 con 0x7fbc50101910 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc568bf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5010c4f0 con 0x7fbc50100720 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc4ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc50100720 0x7fbc50196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.147+0000 7fbc54e5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc501022d0 0x7fbc5019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc50100720 0x7fbc50196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:60028/0 (socket says 172.21.15.92:60028) 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4ffff640 1 -- 172.21.15.92:0/762681996 learned_addr learned my addr 172.21.15.92:0/762681996 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc54e5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc501022d0 0x7fbc5019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37880/0 (socket says 172.21.15.92:37880) 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4f7fe640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc50101910 0x7fbc50196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4ffff640 1 -- 172.21.15.92:0/762681996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc50101910 msgr2=0x7fbc50196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4ffff640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc50101910 0x7fbc50196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4ffff640 1 -- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc501022d0 msgr2=0x7fbc5019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4ffff640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc501022d0 0x7fbc5019e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4ffff640 1 -- 172.21.15.92:0/762681996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5019e690 con 0x7fbc50100720 2024-03-19T01:50:57.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc54e5c640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc501022d0 0x7fbc5019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:50:57.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4ffff640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc50100720 0x7fbc50196a70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc44000c00 tx=0x7fbc4402c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:57.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4d7fa640 1 -- 172.21.15.92:0/762681996 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44041070 con 0x7fbc50100720 2024-03-19T01:50:57.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5019e920 con 0x7fbc50100720 2024-03-19T01:50:57.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4d7fa640 1 -- 172.21.15.92:0/762681996 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44004450 con 0x7fbc50100720 2024-03-19T01:50:57.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc4d7fa640 1 -- 172.21.15.92:0/762681996 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44034440 con 0x7fbc50100720 2024-03-19T01:50:57.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.148+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5019ee10 con 0x7fbc50100720 2024-03-19T01:50:57.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.149+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc50107b10 con 0x7fbc50100720 2024-03-19T01:50:57.152 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.150+0000 7fbc4d7fa640 1 -- 172.21.15.92:0/762681996 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fbc44044070 con 0x7fbc50100720 2024-03-19T01:50:57.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.151+0000 7fbc4d7fa640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbc380a2880 0x7fbc380a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.151+0000 7fbc4f7fe640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbc380a2880 0x7fbc380a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.151+0000 7fbc4d7fa640 1 -- 172.21.15.92:0/762681996 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbc440f0680 con 0x7fbc50100720 2024-03-19T01:50:57.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.151+0000 7fbc4f7fe640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbc380a2880 0x7fbc380a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbc50197f80 tx=0x7fbc40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:57.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.154+0000 7fbc4d7fa640 1 -- 172.21.15.92:0/762681996 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbc440bd210 con 0x7fbc50100720 2024-03-19T01:50:57.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:57 smithi092 ceph-mon[62574]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:57.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:57 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] up:active 2024-03-19T01:50:57.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:57 smithi092 ceph-mon[62574]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-19T01:50:57.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:57 smithi092 ceph-mon[62574]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:50:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:57 smithi144 ceph-mon[67199]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:57 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] up:active 2024-03-19T01:50:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:57 smithi144 ceph-mon[67199]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-19T01:50:57.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:57 smithi144 ceph-mon[67199]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:50:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.363+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fbc50061c60 con 0x7fbc380a2880 2024-03-19T01:50:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.367+0000 7fbc4d7fa640 1 -- 172.21.15.92:0/762681996 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbc50061c60 con 0x7fbc380a2880 2024-03-19T01:50:57.368 INFO:teuthology.orchestra.run.smithi092.stdout:/volumes/qa/sv_1/9e966a37-f1ed-4f65-93aa-7bf90d627330 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbc380a2880 msgr2=0x7fbc380a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbc380a2880 0x7fbc380a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbc50197f80 tx=0x7fbc40005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc50100720 msgr2=0x7fbc50196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc50100720 0x7fbc50196a70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc44000c00 tx=0x7fbc4402c640 comp rx=0 tx=0).stop 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 shutdown_connections 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc501022d0 0x7fbc5019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc50101910 0x7fbc50196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbc380a2880 0x7fbc380a4d40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 --2- 172.21.15.92:0/762681996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc50100720 0x7fbc50196a70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 >> 172.21.15.92:0/762681996 conn(0x7fbc500fbcb0 msgr2=0x7fbc50104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:57.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 shutdown_connections 2024-03-19T01:50:57.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.369+0000 7fbc568bf640 1 -- 172.21.15.92:0/762681996 wait complete. 2024-03-19T01:50:57.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:50:57.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:57 smithi159 ceph-mon[67110]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:57.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:57 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] up:active 2024-03-19T01:50:57.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:57 smithi159 ceph-mon[67110]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-19T01:50:57.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:57 smithi159 ceph-mon[67110]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:50:57.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.482+0000 7f4fa9153640 1 -- 172.21.15.92:0/2851869776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa41022f0 msgr2=0x7f4fa4108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.482+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2851869776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa41022f0 0x7f4fa4108b80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009ca0 tx=0x7f4f9c02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.482+0000 7f4fa9153640 1 -- 172.21.15.92:0/2851869776 shutdown_connections 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.482+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2851869776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa41022f0 0x7f4fa4108b80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.482+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2851869776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa4101930 0x7f4fa4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.482+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2851869776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa4100740 0x7f4fa4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.482+0000 7f4fa9153640 1 -- 172.21.15.92:0/2851869776 >> 172.21.15.92:0/2851869776 conn(0x7f4fa40fbcb0 msgr2=0x7f4fa40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.482+0000 7f4fa9153640 1 -- 172.21.15.92:0/2851869776 shutdown_connections 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.482+0000 7f4fa9153640 1 -- 172.21.15.92:0/2851869776 wait complete. 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa9153640 1 Processor -- start 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa9153640 1 -- start start 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa9153640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa4100740 0x7f4fa4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa9153640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa4101930 0x7f4fa4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa9153640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa41022f0 0x7f4fa419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa9153640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa410d4e0 con 0x7f4fa4101930 2024-03-19T01:50:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa9153640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa410d360 con 0x7f4fa41022f0 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa9153640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa410d660 con 0x7f4fa4100740 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4f9bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa4101930 0x7f4fa4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4f9bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa4101930 0x7f4fa4198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37892/0 (socket says 172.21.15.92:37892) 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4f9bfff640 1 -- 172.21.15.92:0/2999399044 learned_addr learned my addr 172.21.15.92:0/2999399044 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa2d9d640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa4100740 0x7f4fa4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4fa359e640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa41022f0 0x7f4fa419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4f9bfff640 1 -- 172.21.15.92:0/2999399044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa4100740 msgr2=0x7f4fa4197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4f9bfff640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa4100740 0x7f4fa4197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4f9bfff640 1 -- 172.21.15.92:0/2999399044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa41022f0 msgr2=0x7f4fa419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4f9bfff640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa41022f0 0x7f4fa419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.483+0000 7f4f9bfff640 1 -- 172.21.15.92:0/2999399044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa419f730 con 0x7f4fa4101930 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.484+0000 7f4f9bfff640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa4101930 0x7f4fa4198050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00e9f0 tx=0x7f4f8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.484+0000 7f4fa0d99640 1 -- 172.21.15.92:0/2999399044 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00cd80 con 0x7f4fa4101930 2024-03-19T01:50:57.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.484+0000 7f4fa0d99640 1 -- 172.21.15.92:0/2999399044 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c010910 con 0x7f4fa4101930 2024-03-19T01:50:57.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.484+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa419fa20 con 0x7f4fa4101930 2024-03-19T01:50:57.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.484+0000 7f4fa0d99640 1 -- 172.21.15.92:0/2999399044 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c018990 con 0x7f4fa4101930 2024-03-19T01:50:57.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.484+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa419ff70 con 0x7f4fa4101930 2024-03-19T01:50:57.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.485+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa4107a60 con 0x7f4fa4101930 2024-03-19T01:50:57.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.486+0000 7f4fa0d99640 1 -- 172.21.15.92:0/2999399044 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4f8c010ac0 con 0x7f4fa4101930 2024-03-19T01:50:57.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.487+0000 7f4fa0d99640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4f900a2800 0x7f4f900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.487+0000 7f4fa2d9d640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4f900a2800 0x7f4f900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.487+0000 7f4fa0d99640 1 -- 172.21.15.92:0/2999399044 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4f8c0cf020 con 0x7f4fa4101930 2024-03-19T01:50:57.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.487+0000 7f4fa2d9d640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4f900a2800 0x7f4f900a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4101790 tx=0x7f4f94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:57.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.490+0000 7f4fa0d99640 1 -- 172.21.15.92:0/2999399044 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4f8c09bbc0 con 0x7f4fa4101930 2024-03-19T01:50:57.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.711+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fa402d3b0 con 0x7f4fa4101930 2024-03-19T01:50:57.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.712+0000 7f4fa0d99640 1 -- 172.21.15.92:0/2999399044 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11932 (secure 0 0 0) 0x7f4f8c09fc60 con 0x7f4fa4101930 2024-03-19T01:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:57.712 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:50:49.881618+0000","modified":"2024-03-19T01:50:56.972139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24476,"mds_2":24475,"mds_3":24457,"mds_4":14601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6839/380126062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":380126062},{"type":"v1","addr":"172.21.15.144:6839","nonce":380126062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24476,24493,24475,24457,14601]},"id":1}]} 2024-03-19T01:50:57.713 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2024-03-19T01:50:57.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4f900a2800 msgr2=0x7f4f900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4f900a2800 0x7f4f900a4cc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4101790 tx=0x7f4f94005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:57.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa4101930 msgr2=0x7f4fa4198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa4101930 0x7f4fa4198050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00e9f0 tx=0x7f4f8c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 shutdown_connections 2024-03-19T01:50:57.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fa41022f0 0x7f4fa419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa4101930 0x7f4fa4198050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4f900a2800 0x7f4f900a4cc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 --2- 172.21.15.92:0/2999399044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fa4100740 0x7f4fa4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.714+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 >> 172.21.15.92:0/2999399044 conn(0x7f4fa40fbcb0 msgr2=0x7f4fa4104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.715+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 shutdown_connections 2024-03-19T01:50:57.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.715+0000 7f4fa9153640 1 -- 172.21.15.92:0/2999399044 wait complete. 2024-03-19T01:50:57.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:50:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.832+0000 7f9d5159e640 1 -- 172.21.15.92:0/548283703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c101930 msgr2=0x7f9d4c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.832+0000 7f9d5159e640 1 --2- 172.21.15.92:0/548283703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c101930 0x7f9d4c101db0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009900 tx=0x7f9d3402bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.832+0000 7f9d5159e640 1 -- 172.21.15.92:0/548283703 shutdown_connections 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.832+0000 7f9d5159e640 1 --2- 172.21.15.92:0/548283703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d4c1022f0 0x7f9d4c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.832+0000 7f9d5159e640 1 --2- 172.21.15.92:0/548283703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c101930 0x7f9d4c101db0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.832+0000 7f9d5159e640 1 --2- 172.21.15.92:0/548283703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d4c100740 0x7f9d4c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.832+0000 7f9d5159e640 1 -- 172.21.15.92:0/548283703 >> 172.21.15.92:0/548283703 conn(0x7f9d4c0fbcb0 msgr2=0x7f9d4c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.832+0000 7f9d5159e640 1 -- 172.21.15.92:0/548283703 shutdown_connections 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.832+0000 7f9d5159e640 1 -- 172.21.15.92:0/548283703 wait complete. 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d5159e640 1 Processor -- start 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d5159e640 1 -- start start 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d5159e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d4c100740 0x7f9d4c196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d5159e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d4c101930 0x7f9d4c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d5159e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c1022f0 0x7f9d4c19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d5159e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c10c3c0 con 0x7f9d4c1022f0 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d5159e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c10c240 con 0x7f9d4c101930 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d5159e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c10c540 con 0x7f9d4c100740 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c1022f0 0x7f9d4c19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d4c100740 0x7f9d4c196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c1022f0 0x7f9d4c19dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37914/0 (socket says 172.21.15.92:37914) 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4b7fe640 1 -- 172.21.15.92:0/508377502 learned_addr learned my addr 172.21.15.92:0/508377502 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d43fff640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d4c101930 0x7f9d4c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4b7fe640 1 -- 172.21.15.92:0/508377502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d4c100740 msgr2=0x7f9d4c196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4b7fe640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d4c100740 0x7f9d4c196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4b7fe640 1 -- 172.21.15.92:0/508377502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d4c101930 msgr2=0x7f9d4c196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4b7fe640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d4c101930 0x7f9d4c196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4b7fe640 1 -- 172.21.15.92:0/508377502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c19e660 con 0x7f9d4c1022f0 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.833+0000 7f9d4b7fe640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c1022f0 0x7f9d4c19dff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400eb10 tx=0x7f9d4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:57.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.834+0000 7f9d48ff9640 1 -- 172.21.15.92:0/508377502 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44004b20 con 0x7f9d4c1022f0 2024-03-19T01:50:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.834+0000 7f9d48ff9640 1 -- 172.21.15.92:0/508377502 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d44004cc0 con 0x7f9d4c1022f0 2024-03-19T01:50:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.834+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4c19e950 con 0x7f9d4c1022f0 2024-03-19T01:50:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.834+0000 7f9d48ff9640 1 -- 172.21.15.92:0/508377502 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44010760 con 0x7f9d4c1022f0 2024-03-19T01:50:57.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.834+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4c19ee70 con 0x7f9d4c1022f0 2024-03-19T01:50:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.835+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d4c107ae0 con 0x7f9d4c1022f0 2024-03-19T01:50:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.836+0000 7f9d48ff9640 1 -- 172.21.15.92:0/508377502 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f9d44005020 con 0x7f9d4c1022f0 2024-03-19T01:50:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.836+0000 7f9d48ff9640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d380a2880 0x7f9d380a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.836+0000 7f9d48ff9640 1 -- 172.21.15.92:0/508377502 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9d440cedf0 con 0x7f9d4c1022f0 2024-03-19T01:50:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.836+0000 7f9d4affd640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d380a2880 0x7f9d380a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.837+0000 7f9d4affd640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d380a2880 0x7f9d380a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c101790 tx=0x7f9d3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:57.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:57.841+0000 7f9d48ff9640 1 -- 172.21.15.92:0/508377502 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9d4409bfe0 con 0x7f9d4c1022f0 2024-03-19T01:50:58.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.018+0000 7f9d48ff9640 1 -- 172.21.15.92:0/508377502 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9d44097f50 con 0x7f9d4c1022f0 2024-03-19T01:50:58.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.032+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d4c02d3c0 con 0x7f9d4c1022f0 2024-03-19T01:50:58.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.034+0000 7f9d48ff9640 1 -- 172.21.15.92:0/508377502 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22256 (secure 0 0 0) 0x7f9d44046890 con 0x7f9d4c1022f0 2024-03-19T01:50:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:58.035 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":85,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:50:51.542012+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:50:58.036 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d380a2880 msgr2=0x7f9d380a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d380a2880 0x7f9d380a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c101790 tx=0x7f9d3c005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c1022f0 msgr2=0x7f9d4c19dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c1022f0 0x7f9d4c19dff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400eb10 tx=0x7f9d4400c2d0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 shutdown_connections 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c1022f0 0x7f9d4c19dff0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d4c101930 0x7f9d4c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d380a2880 0x7f9d380a4d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 --2- 172.21.15.92:0/508377502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d4c100740 0x7f9d4c196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 >> 172.21.15.92:0/508377502 conn(0x7f9d4c0fbcb0 msgr2=0x7f9d4c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 shutdown_connections 2024-03-19T01:50:58.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.036+0000 7f9d5159e640 1 -- 172.21.15.92:0/508377502 wait complete. 2024-03-19T01:50:58.048 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-19T01:50:58.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.152+0000 7fb17f9b3640 1 -- 172.21.15.92:0/1885603914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 msgr2=0x7fb178107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.152+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/1885603914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 0x7fb178107080 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb168009900 tx=0x7fb16802bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.152+0000 7fb17f9b3640 1 -- 172.21.15.92:0/1885603914 shutdown_connections 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.152+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/1885603914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1781075c0 0x7fb1781099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.152+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/1885603914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 0x7fb178107080 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.152+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/1885603914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178102360 0x7fb178104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.152+0000 7fb17f9b3640 1 -- 172.21.15.92:0/1885603914 >> 172.21.15.92:0/1885603914 conn(0x7fb1780fbcb0 msgr2=0x7fb1780fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 -- 172.21.15.92:0/1885603914 shutdown_connections 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 -- 172.21.15.92:0/1885603914 wait complete. 2024-03-19T01:50:58.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 Processor -- start 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 -- start start 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178102360 0x7fb178197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 0x7fb1781980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1781075c0 0x7fb17819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb17810d490 con 0x7fb178104c90 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb17810d310 con 0x7fb1781075c0 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17f9b3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb17810d610 con 0x7fb178102360 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17cf4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 0x7fb1781980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17d74f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178102360 0x7fb178197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17cf4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 0x7fb1781980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37938/0 (socket says 172.21.15.92:37938) 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.153+0000 7fb17cf4e640 1 -- 172.21.15.92:0/3561660010 learned_addr learned my addr 172.21.15.92:0/3561660010 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17df50640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1781075c0 0x7fb17819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:58.156 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17cf4e640 1 -- 172.21.15.92:0/3561660010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178102360 msgr2=0x7fb178197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17cf4e640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178102360 0x7fb178197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17cf4e640 1 -- 172.21.15.92:0/3561660010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1781075c0 msgr2=0x7fb17819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17cf4e640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1781075c0 0x7fb17819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17cf4e640 1 -- 172.21.15.92:0/3561660010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17819f7b0 con 0x7fb178104c90 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17d74f640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178102360 0x7fb178197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17cf4e640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 0x7fb1781980d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb168000c00 tx=0x7fb16802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb16e7fc640 1 -- 172.21.15.92:0/3561660010 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168041070 con 0x7fb178104c90 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb16e7fc640 1 -- 172.21.15.92:0/3561660010 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb168004450 con 0x7fb178104c90 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17819fa40 con 0x7fb178104c90 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb16e7fc640 1 -- 172.21.15.92:0/3561660010 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168034440 con 0x7fb178104c90 2024-03-19T01:50:58.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.154+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17819ff30 con 0x7fb178104c90 2024-03-19T01:50:58.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.155+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1780fd7a0 con 0x7fb178104c90 2024-03-19T01:50:58.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.156+0000 7fb16e7fc640 1 -- 172.21.15.92:0/3561660010 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb168044070 con 0x7fb178104c90 2024-03-19T01:50:58.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.157+0000 7fb16e7fc640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1640a2890 0x7fb1640a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:58.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.157+0000 7fb17d74f640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1640a2890 0x7fb1640a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:58.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.157+0000 7fb16e7fc640 1 -- 172.21.15.92:0/3561660010 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb1680f03f0 con 0x7fb178104c90 2024-03-19T01:50:58.158 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.157+0000 7fb17d74f640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1640a2890 0x7fb1640a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb170005fd0 tx=0x7fb170009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:58.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.161+0000 7fb16e7fc640 1 -- 172.21.15.92:0/3561660010 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb1680bd030 con 0x7fb178104c90 2024-03-19T01:50:58.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:58 smithi092 ceph-mon[62574]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:58.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:58 smithi092 ceph-mon[62574]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:58.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:58 smithi092 ceph-mon[62574]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:58.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:58 smithi092 ceph-mon[62574]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 646 B/s rd, 4.9 KiB/s wr, 18 op/s 2024-03-19T01:50:58.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:58 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2999399044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:58.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:58 smithi144 ceph-mon[67199]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:58.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:58 smithi144 ceph-mon[67199]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:58.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:58 smithi144 ceph-mon[67199]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:58.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:58 smithi144 ceph-mon[67199]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 646 B/s rd, 4.9 KiB/s wr, 18 op/s 2024-03-19T01:50:58.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:58 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2999399044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:58.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.367+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb178061c60 con 0x7fb1640a2890 2024-03-19T01:50:58.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.379+0000 7fb16e7fc640 1 -- 172.21.15.92:0/3561660010 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258943 (secure 0 0 0) 0x7fb178061c60 con 0x7fb1640a2890 2024-03-19T01:50:58.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:58.384 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-03-19T01:50:58.385 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1640a2890 msgr2=0x7fb1640a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1640a2890 0x7fb1640a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb170005fd0 tx=0x7fb170009500 comp rx=0 tx=0).stop 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 msgr2=0x7fb1781980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 0x7fb1781980d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb168000c00 tx=0x7fb16802c640 comp rx=0 tx=0).stop 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 shutdown_connections 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1781075c0 0x7fb17819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb178104c90 0x7fb1781980d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1640a2890 0x7fb1640a4d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 --2- 172.21.15.92:0/3561660010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178102360 0x7fb178197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.382+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 >> 172.21.15.92:0/3561660010 conn(0x7fb1780fbcb0 msgr2=0x7fb178102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.383+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 shutdown_connections 2024-03-19T01:50:58.386 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.383+0000 7fb17f9b3640 1 -- 172.21.15.92:0/3561660010 wait complete. 2024-03-19T01:50:58.394 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-03-19T01:50:57.446099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595590,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":133,"num_read_kb":114,"num_write":324,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351964,"kb_used_data":11088,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589860,"statfs":{"total":1151940427776,"available":1151580016640,"internally_reserved":0,"allocated":11354112,"data_stored":7066356,"data_compressed":107082,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":348894524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5222,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":143,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.915711"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063395+0000","last_change":"2024-03-19T01:50:50.561781+0000","last_active":"2024-03-19T01:50:52.063395+0000","last_peered":"2024-03-19T01:50:52.063395+0000","last_clean":"2024-03-19T01:50:52.063395+0000","last_became_active":"2024-03-19T01:50:50.561402+0000","last_became_peered":"2024-03-19T01:50:50.561402+0000","last_unstale":"2024-03-19T01:50:52.063395+0000","last_undegraded":"2024-03-19T01:50:52.063395+0000","last_fullsized":"2024-03-19T01:50:52.063395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:07.257394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062355+0000","last_change":"2024-03-19T01:50:50.558625+0000","last_active":"2024-03-19T01:50:52.062355+0000","last_peered":"2024-03-19T01:50:52.062355+0000","last_clean":"2024-03-19T01:50:52.062355+0000","last_became_active":"2024-03-19T01:50:50.557349+0000","last_became_peered":"2024-03-19T01:50:50.557349+0000","last_unstale":"2024-03-19T01:50:52.062355+0000","last_undegraded":"2024-03-19T01:50:52.062355+0000","last_fullsized":"2024-03-19T01:50:52.062355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:05.791529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062322+0000","last_change":"2024-03-19T01:50:50.554103+0000","last_active":"2024-03-19T01:50:52.062322+0000","last_peered":"2024-03-19T01:50:52.062322+0000","last_clean":"2024-03-19T01:50:52.062322+0000","last_became_active":"2024-03-19T01:50:50.553734+0000","last_became_peered":"2024-03-19T01:50:50.553734+0000","last_unstale":"2024-03-19T01:50:52.062322+0000","last_undegraded":"2024-03-19T01:50:52.062322+0000","last_fullsized":"2024-03-19T01:50:52.062322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:18:02.768774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.697597+0000","last_change":"2024-03-19T01:50:50.571980+0000","last_active":"2024-03-19T01:50:51.697597+0000","last_peered":"2024-03-19T01:50:51.697597+0000","last_clean":"2024-03-19T01:50:51.697597+0000","last_became_active":"2024-03-19T01:50:50.571679+0000","last_became_peered":"2024-03-19T01:50:50.571679+0000","last_unstale":"2024-03-19T01:50:51.697597+0000","last_undegraded":"2024-03-19T01:50:51.697597+0000","last_fullsized":"2024-03-19T01:50:51.697597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:30:08.946509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062334+0000","last_change":"2024-03-19T01:50:50.563314+0000","last_active":"2024-03-19T01:50:52.062334+0000","last_peered":"2024-03-19T01:50:52.062334+0000","last_clean":"2024-03-19T01:50:52.062334+0000","last_became_active":"2024-03-19T01:50:50.562953+0000","last_became_peered":"2024-03-19T01:50:50.562953+0000","last_unstale":"2024-03-19T01:50:52.062334+0000","last_undegraded":"2024-03-19T01:50:52.062334+0000","last_fullsized":"2024-03-19T01:50:52.062334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:38.139670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063398+0000","last_change":"2024-03-19T01:50:50.555305+0000","last_active":"2024-03-19T01:50:52.063398+0000","last_peered":"2024-03-19T01:50:52.063398+0000","last_clean":"2024-03-19T01:50:52.063398+0000","last_became_active":"2024-03-19T01:50:50.554930+0000","last_became_peered":"2024-03-19T01:50:50.554930+0000","last_unstale":"2024-03-19T01:50:52.063398+0000","last_undegraded":"2024-03-19T01:50:52.063398+0000","last_fullsized":"2024-03-19T01:50:52.063398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:05:03.418572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062714+0000","last_change":"2024-03-19T01:50:50.557381+0000","last_active":"2024-03-19T01:50:52.062714+0000","last_peered":"2024-03-19T01:50:52.062714+0000","last_clean":"2024-03-19T01:50:52.062714+0000","last_became_active":"2024-03-19T01:50:50.556804+0000","last_became_peered":"2024-03-19T01:50:50.556804+0000","last_unstale":"2024-03-19T01:50:52.062714+0000","last_undegraded":"2024-03-19T01:50:52.062714+0000","last_fullsized":"2024-03-19T01:50:52.062714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:54:00.676606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062622+0000","last_change":"2024-03-19T01:50:50.547526+0000","last_active":"2024-03-19T01:50:52.062622+0000","last_peered":"2024-03-19T01:50:52.062622+0000","last_clean":"2024-03-19T01:50:52.062622+0000","last_became_active":"2024-03-19T01:50:50.547053+0000","last_became_peered":"2024-03-19T01:50:50.547053+0000","last_unstale":"2024-03-19T01:50:52.062622+0000","last_undegraded":"2024-03-19T01:50:52.062622+0000","last_fullsized":"2024-03-19T01:50:52.062622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:17.050026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.552369+0000","last_change":"2024-03-19T01:50:50.562970+0000","last_active":"2024-03-19T01:50:51.552369+0000","last_peered":"2024-03-19T01:50:51.552369+0000","last_clean":"2024-03-19T01:50:51.552369+0000","last_became_active":"2024-03-19T01:50:50.562372+0000","last_became_peered":"2024-03-19T01:50:50.562372+0000","last_unstale":"2024-03-19T01:50:51.552369+0000","last_undegraded":"2024-03-19T01:50:51.552369+0000","last_fullsized":"2024-03-19T01:50:51.552369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:57.861083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063799+0000","last_change":"2024-03-19T01:50:50.559369+0000","last_active":"2024-03-19T01:50:52.063799+0000","last_peered":"2024-03-19T01:50:52.063799+0000","last_clean":"2024-03-19T01:50:52.063799+0000","last_became_active":"2024-03-19T01:50:50.559062+0000","last_became_peered":"2024-03-19T01:50:50.559062+0000","last_unstale":"2024-03-19T01:50:52.063799+0000","last_undegraded":"2024-03-19T01:50:52.063799+0000","last_fullsized":"2024-03-19T01:50:52.063799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:08:22.977566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062876+0000","last_change":"2024-03-19T01:50:50.567000+0000","last_active":"2024-03-19T01:50:52.062876+0000","last_peered":"2024-03-19T01:50:52.062876+0000","last_clean":"2024-03-19T01:50:52.062876+0000","last_became_active":"2024-03-19T01:50:50.566613+0000","last_became_peered":"2024-03-19T01:50:50.566613+0000","last_unstale":"2024-03-19T01:50:52.062876+0000","last_undegraded":"2024-03-19T01:50:52.062876+0000","last_fullsized":"2024-03-19T01:50:52.062876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:40:39.132559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.697589+0000","last_change":"2024-03-19T01:50:50.564354+0000","last_active":"2024-03-19T01:50:51.697589+0000","last_peered":"2024-03-19T01:50:51.697589+0000","last_clean":"2024-03-19T01:50:51.697589+0000","last_became_active":"2024-03-19T01:50:50.563997+0000","last_became_peered":"2024-03-19T01:50:50.563997+0000","last_unstale":"2024-03-19T01:50:51.697589+0000","last_undegraded":"2024-03-19T01:50:51.697589+0000","last_fullsized":"2024-03-19T01:50:51.697589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:27:50.266867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550077+0000","last_change":"2024-03-19T01:50:50.539164+0000","last_active":"2024-03-19T01:50:51.550077+0000","last_peered":"2024-03-19T01:50:51.550077+0000","last_clean":"2024-03-19T01:50:51.550077+0000","last_became_active":"2024-03-19T01:50:50.538792+0000","last_became_peered":"2024-03-19T01:50:50.538792+0000","last_unstale":"2024-03-19T01:50:51.550077+0000","last_undegraded":"2024-03-19T01:50:51.550077+0000","last_fullsized":"2024-03-19T01:50:51.550077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:07:56.700994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062274+0000","last_change":"2024-03-19T01:50:50.567590+0000","last_active":"2024-03-19T01:50:52.062274+0000","last_peered":"2024-03-19T01:50:52.062274+0000","last_clean":"2024-03-19T01:50:52.062274+0000","last_became_active":"2024-03-19T01:50:50.567225+0000","last_became_peered":"2024-03-19T01:50:50.567225+0000","last_unstale":"2024-03-19T01:50:52.062274+0000","last_undegraded":"2024-03-19T01:50:52.062274+0000","last_fullsized":"2024-03-19T01:50:52.062274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:55.537567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062115+0000","last_change":"2024-03-19T01:50:50.571815+0000","last_active":"2024-03-19T01:50:52.062115+0000","last_peered":"2024-03-19T01:50:52.062115+0000","last_clean":"2024-03-19T01:50:52.062115+0000","last_became_active":"2024-03-19T01:50:50.571484+0000","last_became_peered":"2024-03-19T01:50:50.571484+0000","last_unstale":"2024-03-19T01:50:52.062115+0000","last_undegraded":"2024-03-19T01:50:52.062115+0000","last_fullsized":"2024-03-19T01:50:52.062115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:40:12.230817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.551949+0000","last_change":"2024-03-19T01:50:50.558616+0000","last_active":"2024-03-19T01:50:51.551949+0000","last_peered":"2024-03-19T01:50:51.551949+0000","last_clean":"2024-03-19T01:50:51.551949+0000","last_became_active":"2024-03-19T01:50:50.558173+0000","last_became_peered":"2024-03-19T01:50:50.558173+0000","last_unstale":"2024-03-19T01:50:51.551949+0000","last_undegraded":"2024-03-19T01:50:51.551949+0000","last_fullsized":"2024-03-19T01:50:51.551949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:04:47.174828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063149+0000","last_change":"2024-03-19T01:50:50.563450+0000","last_active":"2024-03-19T01:50:52.063149+0000","last_peered":"2024-03-19T01:50:52.063149+0000","last_clean":"2024-03-19T01:50:52.063149+0000","last_became_active":"2024-03-19T01:50:50.562663+0000","last_became_peered":"2024-03-19T01:50:50.562663+0000","last_unstale":"2024-03-19T01:50:52.063149+0000","last_undegraded":"2024-03-19T01:50:52.063149+0000","last_fullsized":"2024-03-19T01:50:52.063149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:57:47.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.840669+0000","last_change":"2024-03-19T01:50:50.559848+0000","last_active":"2024-03-19T01:50:51.840669+0000","last_peered":"2024-03-19T01:50:51.840669+0000","last_clean":"2024-03-19T01:50:51.840669+0000","last_became_active":"2024-03-19T01:50:50.559515+0000","last_became_peered":"2024-03-19T01:50:50.559515+0000","last_unstale":"2024-03-19T01:50:51.840669+0000","last_undegraded":"2024-03-19T01:50:51.840669+0000","last_fullsized":"2024-03-19T01:50:51.840669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:16:15.262613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549838+0000","last_change":"2024-03-19T01:50:50.559994+0000","last_active":"2024-03-19T01:50:51.549838+0000","last_peered":"2024-03-19T01:50:51.549838+0000","last_clean":"2024-03-19T01:50:51.549838+0000","last_became_active":"2024-03-19T01:50:50.558759+0000","last_became_peered":"2024-03-19T01:50:50.558759+0000","last_unstale":"2024-03-19T01:50:51.549838+0000","last_undegraded":"2024-03-19T01:50:51.549838+0000","last_fullsized":"2024-03-19T01:50:51.549838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:53.957096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.974634+0000","last_change":"2024-03-19T01:50:49.550130+0000","last_active":"2024-03-19T01:50:55.974634+0000","last_peered":"2024-03-19T01:50:55.974634+0000","last_clean":"2024-03-19T01:50:55.974634+0000","last_became_active":"2024-03-19T01:50:49.549374+0000","last_became_peered":"2024-03-19T01:50:49.549374+0000","last_unstale":"2024-03-19T01:50:55.974634+0000","last_undegraded":"2024-03-19T01:50:55.974634+0000","last_fullsized":"2024-03-19T01:50:55.974634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:14:43.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063109+0000","last_change":"2024-03-19T01:50:50.570319+0000","last_active":"2024-03-19T01:50:52.063109+0000","last_peered":"2024-03-19T01:50:52.063109+0000","last_clean":"2024-03-19T01:50:52.063109+0000","last_became_active":"2024-03-19T01:50:50.569937+0000","last_became_peered":"2024-03-19T01:50:50.569937+0000","last_unstale":"2024-03-19T01:50:52.063109+0000","last_undegraded":"2024-03-19T01:50:52.063109+0000","last_fullsized":"2024-03-19T01:50:52.063109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:38:00.024688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063239+0000","last_change":"2024-03-19T01:50:49.551261+0000","last_active":"2024-03-19T01:50:52.063239+0000","last_peered":"2024-03-19T01:50:52.063239+0000","last_clean":"2024-03-19T01:50:52.063239+0000","last_became_active":"2024-03-19T01:50:49.550949+0000","last_became_peered":"2024-03-19T01:50:49.550949+0000","last_unstale":"2024-03-19T01:50:52.063239+0000","last_undegraded":"2024-03-19T01:50:52.063239+0000","last_fullsized":"2024-03-19T01:50:52.063239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:23:49.064155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062459+0000","last_change":"2024-03-19T01:50:50.554303+0000","last_active":"2024-03-19T01:50:52.062459+0000","last_peered":"2024-03-19T01:50:52.062459+0000","last_clean":"2024-03-19T01:50:52.062459+0000","last_became_active":"2024-03-19T01:50:50.553765+0000","last_became_peered":"2024-03-19T01:50:50.553765+0000","last_unstale":"2024-03-19T01:50:52.062459+0000","last_undegraded":"2024-03-19T01:50:52.062459+0000","last_fullsized":"2024-03-19T01:50:52.062459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:42.246266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.004044+0000","last_change":"2024-03-19T01:50:49.554688+0000","last_active":"2024-03-19T01:50:55.004044+0000","last_peered":"2024-03-19T01:50:55.004044+0000","last_clean":"2024-03-19T01:50:55.004044+0000","last_became_active":"2024-03-19T01:50:49.554289+0000","last_became_peered":"2024-03-19T01:50:49.554289+0000","last_unstale":"2024-03-19T01:50:55.004044+0000","last_undegraded":"2024-03-19T01:50:55.004044+0000","last_fullsized":"2024-03-19T01:50:55.004044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:13.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.551911+0000","last_change":"2024-03-19T01:50:50.563305+0000","last_active":"2024-03-19T01:50:51.551911+0000","last_peered":"2024-03-19T01:50:51.551911+0000","last_clean":"2024-03-19T01:50:51.551911+0000","last_became_active":"2024-03-19T01:50:50.561819+0000","last_became_peered":"2024-03-19T01:50:50.561819+0000","last_unstale":"2024-03-19T01:50:51.551911+0000","last_undegraded":"2024-03-19T01:50:51.551911+0000","last_fullsized":"2024-03-19T01:50:51.551911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:11:56.175649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063344+0000","last_change":"2024-03-19T01:50:49.552522+0000","last_active":"2024-03-19T01:50:52.063344+0000","last_peered":"2024-03-19T01:50:52.063344+0000","last_clean":"2024-03-19T01:50:52.063344+0000","last_became_active":"2024-03-19T01:50:49.552083+0000","last_became_peered":"2024-03-19T01:50:49.552083+0000","last_unstale":"2024-03-19T01:50:52.063344+0000","last_undegraded":"2024-03-19T01:50:52.063344+0000","last_fullsized":"2024-03-19T01:50:52.063344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:51:50.069723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063168+0000","last_change":"2024-03-19T01:50:50.560178+0000","last_active":"2024-03-19T01:50:52.063168+0000","last_peered":"2024-03-19T01:50:52.063168+0000","last_clean":"2024-03-19T01:50:52.063168+0000","last_became_active":"2024-03-19T01:50:50.559853+0000","last_became_peered":"2024-03-19T01:50:50.559853+0000","last_unstale":"2024-03-19T01:50:52.063168+0000","last_undegraded":"2024-03-19T01:50:52.063168+0000","last_fullsized":"2024-03-19T01:50:52.063168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:10:04.812207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.993399+0000","last_change":"2024-03-19T01:50:49.550194+0000","last_active":"2024-03-19T01:50:52.993399+0000","last_peered":"2024-03-19T01:50:52.993399+0000","last_clean":"2024-03-19T01:50:52.993399+0000","last_became_active":"2024-03-19T01:50:49.549801+0000","last_became_peered":"2024-03-19T01:50:49.549801+0000","last_unstale":"2024-03-19T01:50:52.993399+0000","last_undegraded":"2024-03-19T01:50:52.993399+0000","last_fullsized":"2024-03-19T01:50:52.993399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:43.110159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062323+0000","last_change":"2024-03-19T01:50:50.560913+0000","last_active":"2024-03-19T01:50:52.062323+0000","last_peered":"2024-03-19T01:50:52.062323+0000","last_clean":"2024-03-19T01:50:52.062323+0000","last_became_active":"2024-03-19T01:50:50.559210+0000","last_became_peered":"2024-03-19T01:50:50.559210+0000","last_unstale":"2024-03-19T01:50:52.062323+0000","last_undegraded":"2024-03-19T01:50:52.062323+0000","last_fullsized":"2024-03-19T01:50:52.062323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:14.814374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063151+0000","last_change":"2024-03-19T01:50:49.548778+0000","last_active":"2024-03-19T01:50:52.063151+0000","last_peered":"2024-03-19T01:50:52.063151+0000","last_clean":"2024-03-19T01:50:52.063151+0000","last_became_active":"2024-03-19T01:50:49.548414+0000","last_became_peered":"2024-03-19T01:50:49.548414+0000","last_unstale":"2024-03-19T01:50:52.063151+0000","last_undegraded":"2024-03-19T01:50:52.063151+0000","last_fullsized":"2024-03-19T01:50:52.063151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:03:37.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063650+0000","last_change":"2024-03-19T01:50:50.559242+0000","last_active":"2024-03-19T01:50:52.063650+0000","last_peered":"2024-03-19T01:50:52.063650+0000","last_clean":"2024-03-19T01:50:52.063650+0000","last_became_active":"2024-03-19T01:50:50.558904+0000","last_became_peered":"2024-03-19T01:50:50.558904+0000","last_unstale":"2024-03-19T01:50:52.063650+0000","last_undegraded":"2024-03-19T01:50:52.063650+0000","last_fullsized":"2024-03-19T01:50:52.063650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:59.249180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549849+0000","last_change":"2024-03-19T01:50:49.548331+0000","last_active":"2024-03-19T01:50:51.549849+0000","last_peered":"2024-03-19T01:50:51.549849+0000","last_clean":"2024-03-19T01:50:51.549849+0000","last_became_active":"2024-03-19T01:50:49.547993+0000","last_became_peered":"2024-03-19T01:50:49.547993+0000","last_unstale":"2024-03-19T01:50:51.549849+0000","last_undegraded":"2024-03-19T01:50:51.549849+0000","last_fullsized":"2024-03-19T01:50:51.549849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:30.059133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062578+0000","last_change":"2024-03-19T01:50:50.553166+0000","last_active":"2024-03-19T01:50:52.062578+0000","last_peered":"2024-03-19T01:50:52.062578+0000","last_clean":"2024-03-19T01:50:52.062578+0000","last_became_active":"2024-03-19T01:50:50.552520+0000","last_became_peered":"2024-03-19T01:50:50.552520+0000","last_unstale":"2024-03-19T01:50:52.062578+0000","last_undegraded":"2024-03-19T01:50:52.062578+0000","last_fullsized":"2024-03-19T01:50:52.062578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:10:24.515191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991005+0000","last_change":"2024-03-19T01:50:49.548868+0000","last_active":"2024-03-19T01:50:52.991005+0000","last_peered":"2024-03-19T01:50:52.991005+0000","last_clean":"2024-03-19T01:50:52.991005+0000","last_became_active":"2024-03-19T01:50:49.548548+0000","last_became_peered":"2024-03-19T01:50:49.548548+0000","last_unstale":"2024-03-19T01:50:52.991005+0000","last_undegraded":"2024-03-19T01:50:52.991005+0000","last_fullsized":"2024-03-19T01:50:52.991005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:21:36.920067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.840577+0000","last_change":"2024-03-19T01:50:50.559870+0000","last_active":"2024-03-19T01:50:51.840577+0000","last_peered":"2024-03-19T01:50:51.840577+0000","last_clean":"2024-03-19T01:50:51.840577+0000","last_became_active":"2024-03-19T01:50:50.559530+0000","last_became_peered":"2024-03-19T01:50:50.559530+0000","last_unstale":"2024-03-19T01:50:51.840577+0000","last_undegraded":"2024-03-19T01:50:51.840577+0000","last_fullsized":"2024-03-19T01:50:51.840577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:39:48.635416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991521+0000","last_change":"2024-03-19T01:50:49.554733+0000","last_active":"2024-03-19T01:50:52.991521+0000","last_peered":"2024-03-19T01:50:52.991521+0000","last_clean":"2024-03-19T01:50:52.991521+0000","last_became_active":"2024-03-19T01:50:49.554366+0000","last_became_peered":"2024-03-19T01:50:49.554366+0000","last_unstale":"2024-03-19T01:50:52.991521+0000","last_undegraded":"2024-03-19T01:50:52.991521+0000","last_fullsized":"2024-03-19T01:50:52.991521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:53:03.790255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.552222+0000","last_change":"2024-03-19T01:50:50.561526+0000","last_active":"2024-03-19T01:50:51.552222+0000","last_peered":"2024-03-19T01:50:51.552222+0000","last_clean":"2024-03-19T01:50:51.552222+0000","last_became_active":"2024-03-19T01:50:50.561189+0000","last_became_peered":"2024-03-19T01:50:50.561189+0000","last_unstale":"2024-03-19T01:50:51.552222+0000","last_undegraded":"2024-03-19T01:50:51.552222+0000","last_fullsized":"2024-03-19T01:50:51.552222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:15:18.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991239+0000","last_change":"2024-03-19T01:50:49.553701+0000","last_active":"2024-03-19T01:50:52.991239+0000","last_peered":"2024-03-19T01:50:52.991239+0000","last_clean":"2024-03-19T01:50:52.991239+0000","last_became_active":"2024-03-19T01:50:49.553079+0000","last_became_peered":"2024-03-19T01:50:49.553079+0000","last_unstale":"2024-03-19T01:50:52.991239+0000","last_undegraded":"2024-03-19T01:50:52.991239+0000","last_fullsized":"2024-03-19T01:50:52.991239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:19.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062491+0000","last_change":"2024-03-19T01:50:50.565480+0000","last_active":"2024-03-19T01:50:52.062491+0000","last_peered":"2024-03-19T01:50:52.062491+0000","last_clean":"2024-03-19T01:50:52.062491+0000","last_became_active":"2024-03-19T01:50:50.563930+0000","last_became_peered":"2024-03-19T01:50:50.563930+0000","last_unstale":"2024-03-19T01:50:52.062491+0000","last_undegraded":"2024-03-19T01:50:52.062491+0000","last_fullsized":"2024-03-19T01:50:52.062491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:58:55.462003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991869+0000","last_change":"2024-03-19T01:50:49.553573+0000","last_active":"2024-03-19T01:50:52.991869+0000","last_peered":"2024-03-19T01:50:52.991869+0000","last_clean":"2024-03-19T01:50:52.991869+0000","last_became_active":"2024-03-19T01:50:49.553209+0000","last_became_peered":"2024-03-19T01:50:49.553209+0000","last_unstale":"2024-03-19T01:50:52.991869+0000","last_undegraded":"2024-03-19T01:50:52.991869+0000","last_fullsized":"2024-03-19T01:50:52.991869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:47:43.235323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.552095+0000","last_change":"2024-03-19T01:50:50.568133+0000","last_active":"2024-03-19T01:50:51.552095+0000","last_peered":"2024-03-19T01:50:51.552095+0000","last_clean":"2024-03-19T01:50:51.552095+0000","last_became_active":"2024-03-19T01:50:50.567783+0000","last_became_peered":"2024-03-19T01:50:50.567783+0000","last_unstale":"2024-03-19T01:50:51.552095+0000","last_undegraded":"2024-03-19T01:50:51.552095+0000","last_fullsized":"2024-03-19T01:50:51.552095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:16.077111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991348+0000","last_change":"2024-03-19T01:50:49.546463+0000","last_active":"2024-03-19T01:50:52.991348+0000","last_peered":"2024-03-19T01:50:52.991348+0000","last_clean":"2024-03-19T01:50:52.991348+0000","last_became_active":"2024-03-19T01:50:49.546033+0000","last_became_peered":"2024-03-19T01:50:49.546033+0000","last_unstale":"2024-03-19T01:50:52.991348+0000","last_undegraded":"2024-03-19T01:50:52.991348+0000","last_fullsized":"2024-03-19T01:50:52.991348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:29:13.933301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063502+0000","last_change":"2024-03-19T01:50:50.557576+0000","last_active":"2024-03-19T01:50:52.063502+0000","last_peered":"2024-03-19T01:50:52.063502+0000","last_clean":"2024-03-19T01:50:52.063502+0000","last_became_active":"2024-03-19T01:50:50.555024+0000","last_became_peered":"2024-03-19T01:50:50.555024+0000","last_unstale":"2024-03-19T01:50:52.063502+0000","last_undegraded":"2024-03-19T01:50:52.063502+0000","last_fullsized":"2024-03-19T01:50:52.063502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:17.252079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.978230+0000","last_change":"2024-03-19T01:50:49.547209+0000","last_active":"2024-03-19T01:50:55.978230+0000","last_peered":"2024-03-19T01:50:55.978230+0000","last_clean":"2024-03-19T01:50:55.978230+0000","last_became_active":"2024-03-19T01:50:49.546816+0000","last_became_peered":"2024-03-19T01:50:49.546816+0000","last_unstale":"2024-03-19T01:50:55.978230+0000","last_undegraded":"2024-03-19T01:50:55.978230+0000","last_fullsized":"2024-03-19T01:50:55.978230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:42.487614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549965+0000","last_change":"2024-03-19T01:50:50.554679+0000","last_active":"2024-03-19T01:50:51.549965+0000","last_peered":"2024-03-19T01:50:51.549965+0000","last_clean":"2024-03-19T01:50:51.549965+0000","last_became_active":"2024-03-19T01:50:50.554361+0000","last_became_peered":"2024-03-19T01:50:50.554361+0000","last_unstale":"2024-03-19T01:50:51.549965+0000","last_undegraded":"2024-03-19T01:50:51.549965+0000","last_fullsized":"2024-03-19T01:50:51.549965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:44:53.385907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991739+0000","last_change":"2024-03-19T01:50:49.547590+0000","last_active":"2024-03-19T01:50:52.991739+0000","last_peered":"2024-03-19T01:50:52.991739+0000","last_clean":"2024-03-19T01:50:52.991739+0000","last_became_active":"2024-03-19T01:50:49.547261+0000","last_became_peered":"2024-03-19T01:50:49.547261+0000","last_unstale":"2024-03-19T01:50:52.991739+0000","last_undegraded":"2024-03-19T01:50:52.991739+0000","last_fullsized":"2024-03-19T01:50:52.991739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:47.353161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063173+0000","last_change":"2024-03-19T01:50:50.554830+0000","last_active":"2024-03-19T01:50:52.063173+0000","last_peered":"2024-03-19T01:50:52.063173+0000","last_clean":"2024-03-19T01:50:52.063173+0000","last_became_active":"2024-03-19T01:50:50.554495+0000","last_became_peered":"2024-03-19T01:50:50.554495+0000","last_unstale":"2024-03-19T01:50:52.063173+0000","last_undegraded":"2024-03-19T01:50:52.063173+0000","last_fullsized":"2024-03-19T01:50:52.063173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:44.098279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.697176+0000","last_change":"2024-03-19T01:50:49.550056+0000","last_active":"2024-03-19T01:50:51.697176+0000","last_peered":"2024-03-19T01:50:51.697176+0000","last_clean":"2024-03-19T01:50:51.697176+0000","last_became_active":"2024-03-19T01:50:49.549404+0000","last_became_peered":"2024-03-19T01:50:49.549404+0000","last_unstale":"2024-03-19T01:50:51.697176+0000","last_undegraded":"2024-03-19T01:50:51.697176+0000","last_fullsized":"2024-03-19T01:50:51.697176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:38:38.656100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062545+0000","last_change":"2024-03-19T01:50:50.559443+0000","last_active":"2024-03-19T01:50:52.062545+0000","last_peered":"2024-03-19T01:50:52.062545+0000","last_clean":"2024-03-19T01:50:52.062545+0000","last_became_active":"2024-03-19T01:50:50.558992+0000","last_became_peered":"2024-03-19T01:50:50.558992+0000","last_unstale":"2024-03-19T01:50:52.062545+0000","last_undegraded":"2024-03-19T01:50:52.062545+0000","last_fullsized":"2024-03-19T01:50:52.062545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:06:52.144816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:53.998770+0000","last_change":"2024-03-19T01:50:49.548423+0000","last_active":"2024-03-19T01:50:53.998770+0000","last_peered":"2024-03-19T01:50:53.998770+0000","last_clean":"2024-03-19T01:50:53.998770+0000","last_became_active":"2024-03-19T01:50:49.548071+0000","last_became_peered":"2024-03-19T01:50:49.548071+0000","last_unstale":"2024-03-19T01:50:53.998770+0000","last_undegraded":"2024-03-19T01:50:53.998770+0000","last_fullsized":"2024-03-19T01:50:53.998770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:06:00.789358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062464+0000","last_change":"2024-03-19T01:50:50.558569+0000","last_active":"2024-03-19T01:50:52.062464+0000","last_peered":"2024-03-19T01:50:52.062464+0000","last_clean":"2024-03-19T01:50:52.062464+0000","last_became_active":"2024-03-19T01:50:50.557991+0000","last_became_peered":"2024-03-19T01:50:50.557991+0000","last_unstale":"2024-03-19T01:50:52.062464+0000","last_undegraded":"2024-03-19T01:50:52.062464+0000","last_fullsized":"2024-03-19T01:50:52.062464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:55:07.199605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063368+0000","last_change":"2024-03-19T01:50:49.552491+0000","last_active":"2024-03-19T01:50:52.063368+0000","last_peered":"2024-03-19T01:50:52.063368+0000","last_clean":"2024-03-19T01:50:52.063368+0000","last_became_active":"2024-03-19T01:50:49.552133+0000","last_became_peered":"2024-03-19T01:50:49.552133+0000","last_unstale":"2024-03-19T01:50:52.063368+0000","last_undegraded":"2024-03-19T01:50:52.063368+0000","last_fullsized":"2024-03-19T01:50:52.063368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:44.458968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063310+0000","last_change":"2024-03-19T01:50:50.566765+0000","last_active":"2024-03-19T01:50:52.063310+0000","last_peered":"2024-03-19T01:50:52.063310+0000","last_clean":"2024-03-19T01:50:52.063310+0000","last_became_active":"2024-03-19T01:50:50.566412+0000","last_became_peered":"2024-03-19T01:50:50.566412+0000","last_unstale":"2024-03-19T01:50:52.063310+0000","last_undegraded":"2024-03-19T01:50:52.063310+0000","last_fullsized":"2024-03-19T01:50:52.063310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:11:54.418614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.840626+0000","last_change":"2024-03-19T01:50:49.549586+0000","last_active":"2024-03-19T01:50:51.840626+0000","last_peered":"2024-03-19T01:50:51.840626+0000","last_clean":"2024-03-19T01:50:51.840626+0000","last_became_active":"2024-03-19T01:50:49.549224+0000","last_became_peered":"2024-03-19T01:50:49.549224+0000","last_unstale":"2024-03-19T01:50:51.840626+0000","last_undegraded":"2024-03-19T01:50:51.840626+0000","last_fullsized":"2024-03-19T01:50:51.840626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:24:30.591771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063627+0000","last_change":"2024-03-19T01:50:50.554120+0000","last_active":"2024-03-19T01:50:52.063627+0000","last_peered":"2024-03-19T01:50:52.063627+0000","last_clean":"2024-03-19T01:50:52.063627+0000","last_became_active":"2024-03-19T01:50:50.553636+0000","last_became_peered":"2024-03-19T01:50:50.553636+0000","last_unstale":"2024-03-19T01:50:52.063627+0000","last_undegraded":"2024-03-19T01:50:52.063627+0000","last_fullsized":"2024-03-19T01:50:52.063627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:27.634688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063446+0000","last_change":"2024-03-19T01:50:49.551532+0000","last_active":"2024-03-19T01:50:52.063446+0000","last_peered":"2024-03-19T01:50:52.063446+0000","last_clean":"2024-03-19T01:50:52.063446+0000","last_became_active":"2024-03-19T01:50:49.551060+0000","last_became_peered":"2024-03-19T01:50:49.551060+0000","last_unstale":"2024-03-19T01:50:52.063446+0000","last_undegraded":"2024-03-19T01:50:52.063446+0000","last_fullsized":"2024-03-19T01:50:52.063446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:00.339785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063563+0000","last_change":"2024-03-19T01:50:50.556606+0000","last_active":"2024-03-19T01:50:52.063563+0000","last_peered":"2024-03-19T01:50:52.063563+0000","last_clean":"2024-03-19T01:50:52.063563+0000","last_became_active":"2024-03-19T01:50:50.556089+0000","last_became_peered":"2024-03-19T01:50:50.556089+0000","last_unstale":"2024-03-19T01:50:52.063563+0000","last_undegraded":"2024-03-19T01:50:52.063563+0000","last_fullsized":"2024-03-19T01:50:52.063563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:24.256326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062902+0000","last_change":"2024-03-19T01:50:50.564926+0000","last_active":"2024-03-19T01:50:52.062902+0000","last_peered":"2024-03-19T01:50:52.062902+0000","last_clean":"2024-03-19T01:50:52.062902+0000","last_became_active":"2024-03-19T01:50:50.564612+0000","last_became_peered":"2024-03-19T01:50:50.564612+0000","last_unstale":"2024-03-19T01:50:52.062902+0000","last_undegraded":"2024-03-19T01:50:52.062902+0000","last_fullsized":"2024-03-19T01:50:52.062902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:42:29.159512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549788+0000","last_change":"2024-03-19T01:50:49.553780+0000","last_active":"2024-03-19T01:50:51.549788+0000","last_peered":"2024-03-19T01:50:51.549788+0000","last_clean":"2024-03-19T01:50:51.549788+0000","last_became_active":"2024-03-19T01:50:49.552795+0000","last_became_peered":"2024-03-19T01:50:49.552795+0000","last_unstale":"2024-03-19T01:50:51.549788+0000","last_undegraded":"2024-03-19T01:50:51.549788+0000","last_fullsized":"2024-03-19T01:50:51.549788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:01:37.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063059+0000","last_change":"2024-03-19T01:50:50.556943+0000","last_active":"2024-03-19T01:50:52.063059+0000","last_peered":"2024-03-19T01:50:52.063059+0000","last_clean":"2024-03-19T01:50:52.063059+0000","last_became_active":"2024-03-19T01:50:50.556646+0000","last_became_peered":"2024-03-19T01:50:50.556646+0000","last_unstale":"2024-03-19T01:50:52.063059+0000","last_undegraded":"2024-03-19T01:50:52.063059+0000","last_fullsized":"2024-03-19T01:50:52.063059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:59:28.712332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991963+0000","last_change":"2024-03-19T01:50:49.549760+0000","last_active":"2024-03-19T01:50:52.991963+0000","last_peered":"2024-03-19T01:50:52.991963+0000","last_clean":"2024-03-19T01:50:52.991963+0000","last_became_active":"2024-03-19T01:50:49.549218+0000","last_became_peered":"2024-03-19T01:50:49.549218+0000","last_unstale":"2024-03-19T01:50:52.991963+0000","last_undegraded":"2024-03-19T01:50:52.991963+0000","last_fullsized":"2024-03-19T01:50:52.991963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:30:00.802902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062980+0000","last_change":"2024-03-19T01:49:36.469079+0000","last_active":"2024-03-19T01:50:52.062980+0000","last_peered":"2024-03-19T01:50:52.062980+0000","last_clean":"2024-03-19T01:50:52.062980+0000","last_became_active":"2024-03-19T01:49:36.468624+0000","last_became_peered":"2024-03-19T01:49:36.468624+0000","last_unstale":"2024-03-19T01:50:52.062980+0000","last_undegraded":"2024-03-19T01:50:52.062980+0000","last_fullsized":"2024-03-19T01:50:52.062980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:46:07.695833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:46:07.695833+0000","last_clean_scrub_stamp":"2024-03-19T01:46:07.695833+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:06:31.554853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549872+0000","last_change":"2024-03-19T01:50:50.550851+0000","last_active":"2024-03-19T01:50:51.549872+0000","last_peered":"2024-03-19T01:50:51.549872+0000","last_clean":"2024-03-19T01:50:51.549872+0000","last_became_active":"2024-03-19T01:50:50.550581+0000","last_became_peered":"2024-03-19T01:50:50.550581+0000","last_unstale":"2024-03-19T01:50:51.549872+0000","last_undegraded":"2024-03-19T01:50:51.549872+0000","last_fullsized":"2024-03-19T01:50:51.549872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:24:24.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:53.995865+0000","last_change":"2024-03-19T01:50:49.554043+0000","last_active":"2024-03-19T01:50:53.995865+0000","last_peered":"2024-03-19T01:50:53.995865+0000","last_clean":"2024-03-19T01:50:53.995865+0000","last_became_active":"2024-03-19T01:50:49.553597+0000","last_became_peered":"2024-03-19T01:50:49.553597+0000","last_unstale":"2024-03-19T01:50:53.995865+0000","last_undegraded":"2024-03-19T01:50:53.995865+0000","last_fullsized":"2024-03-19T01:50:53.995865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:41:40.545382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.004210+0000","last_change":"2024-03-19T01:50:49.553740+0000","last_active":"2024-03-19T01:50:55.004210+0000","last_peered":"2024-03-19T01:50:55.004210+0000","last_clean":"2024-03-19T01:50:55.004210+0000","last_became_active":"2024-03-19T01:50:49.553230+0000","last_became_peered":"2024-03-19T01:50:49.553230+0000","last_unstale":"2024-03-19T01:50:55.004210+0000","last_undegraded":"2024-03-19T01:50:55.004210+0000","last_fullsized":"2024-03-19T01:50:55.004210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:46:36.633880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550257+0000","last_change":"2024-03-19T01:50:50.554478+0000","last_active":"2024-03-19T01:50:51.550257+0000","last_peered":"2024-03-19T01:50:51.550257+0000","last_clean":"2024-03-19T01:50:51.550257+0000","last_became_active":"2024-03-19T01:50:50.551170+0000","last_became_peered":"2024-03-19T01:50:50.551170+0000","last_unstale":"2024-03-19T01:50:51.550257+0000","last_undegraded":"2024-03-19T01:50:51.550257+0000","last_fullsized":"2024-03-19T01:50:51.550257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:08:12.382239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.005028+0000","last_change":"2024-03-19T01:50:49.554018+0000","last_active":"2024-03-19T01:50:55.005028+0000","last_peered":"2024-03-19T01:50:55.005028+0000","last_clean":"2024-03-19T01:50:55.005028+0000","last_became_active":"2024-03-19T01:50:49.553702+0000","last_became_peered":"2024-03-19T01:50:49.553702+0000","last_unstale":"2024-03-19T01:50:55.005028+0000","last_undegraded":"2024-03-19T01:50:55.005028+0000","last_fullsized":"2024-03-19T01:50:55.005028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:55.051901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063113+0000","last_change":"2024-03-19T01:50:50.565955+0000","last_active":"2024-03-19T01:50:52.063113+0000","last_peered":"2024-03-19T01:50:52.063113+0000","last_clean":"2024-03-19T01:50:52.063113+0000","last_became_active":"2024-03-19T01:50:50.565643+0000","last_became_peered":"2024-03-19T01:50:50.565643+0000","last_unstale":"2024-03-19T01:50:52.063113+0000","last_undegraded":"2024-03-19T01:50:52.063113+0000","last_fullsized":"2024-03-19T01:50:52.063113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:37:42.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.990133+0000","last_change":"2024-03-19T01:50:49.548298+0000","last_active":"2024-03-19T01:50:52.990133+0000","last_peered":"2024-03-19T01:50:52.990133+0000","last_clean":"2024-03-19T01:50:52.990133+0000","last_became_active":"2024-03-19T01:50:49.547941+0000","last_became_peered":"2024-03-19T01:50:49.547941+0000","last_unstale":"2024-03-19T01:50:52.990133+0000","last_undegraded":"2024-03-19T01:50:52.990133+0000","last_fullsized":"2024-03-19T01:50:52.990133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:23:59.221152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062359+0000","last_change":"2024-03-19T01:50:50.563923+0000","last_active":"2024-03-19T01:50:52.062359+0000","last_peered":"2024-03-19T01:50:52.062359+0000","last_clean":"2024-03-19T01:50:52.062359+0000","last_became_active":"2024-03-19T01:50:50.563295+0000","last_became_peered":"2024-03-19T01:50:50.563295+0000","last_unstale":"2024-03-19T01:50:52.062359+0000","last_undegraded":"2024-03-19T01:50:52.062359+0000","last_fullsized":"2024-03-19T01:50:52.062359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:09:47.089720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.990873+0000","last_change":"2024-03-19T01:50:49.548014+0000","last_active":"2024-03-19T01:50:52.990873+0000","last_peered":"2024-03-19T01:50:52.990873+0000","last_clean":"2024-03-19T01:50:52.990873+0000","last_became_active":"2024-03-19T01:50:49.547588+0000","last_became_peered":"2024-03-19T01:50:49.547588+0000","last_unstale":"2024-03-19T01:50:52.990873+0000","last_undegraded":"2024-03-19T01:50:52.990873+0000","last_fullsized":"2024-03-19T01:50:52.990873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:08.258554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062185+0000","last_change":"2024-03-19T01:50:50.562437+0000","last_active":"2024-03-19T01:50:52.062185+0000","last_peered":"2024-03-19T01:50:52.062185+0000","last_clean":"2024-03-19T01:50:52.062185+0000","last_became_active":"2024-03-19T01:50:50.562113+0000","last_became_peered":"2024-03-19T01:50:50.562113+0000","last_unstale":"2024-03-19T01:50:52.062185+0000","last_undegraded":"2024-03-19T01:50:52.062185+0000","last_fullsized":"2024-03-19T01:50:52.062185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:19:31.932037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.976249+0000","last_change":"2024-03-19T01:50:49.547190+0000","last_active":"2024-03-19T01:50:55.976249+0000","last_peered":"2024-03-19T01:50:55.976249+0000","last_clean":"2024-03-19T01:50:55.976249+0000","last_became_active":"2024-03-19T01:50:49.546698+0000","last_became_peered":"2024-03-19T01:50:49.546698+0000","last_unstale":"2024-03-19T01:50:55.976249+0000","last_undegraded":"2024-03-19T01:50:55.976249+0000","last_fullsized":"2024-03-19T01:50:55.976249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:09.557035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062919+0000","last_change":"2024-03-19T01:50:50.571159+0000","last_active":"2024-03-19T01:50:52.062919+0000","last_peered":"2024-03-19T01:50:52.062919+0000","last_clean":"2024-03-19T01:50:52.062919+0000","last_became_active":"2024-03-19T01:50:50.570312+0000","last_became_peered":"2024-03-19T01:50:50.570312+0000","last_unstale":"2024-03-19T01:50:52.062919+0000","last_undegraded":"2024-03-19T01:50:52.062919+0000","last_fullsized":"2024-03-19T01:50:52.062919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:23:16.820114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.003523+0000","last_change":"2024-03-19T01:50:49.550557+0000","last_active":"2024-03-19T01:50:55.003523+0000","last_peered":"2024-03-19T01:50:55.003523+0000","last_clean":"2024-03-19T01:50:55.003523+0000","last_became_active":"2024-03-19T01:50:49.549789+0000","last_became_peered":"2024-03-19T01:50:49.549789+0000","last_unstale":"2024-03-19T01:50:55.003523+0000","last_undegraded":"2024-03-19T01:50:55.003523+0000","last_fullsized":"2024-03-19T01:50:55.003523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:06:14.292689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063001+0000","last_change":"2024-03-19T01:50:50.560116+0000","last_active":"2024-03-19T01:50:52.063001+0000","last_peered":"2024-03-19T01:50:52.063001+0000","last_clean":"2024-03-19T01:50:52.063001+0000","last_became_active":"2024-03-19T01:50:50.559809+0000","last_became_peered":"2024-03-19T01:50:50.559809+0000","last_unstale":"2024-03-19T01:50:52.063001+0000","last_undegraded":"2024-03-19T01:50:52.063001+0000","last_fullsized":"2024-03-19T01:50:52.063001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:26.854262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991448+0000","last_change":"2024-03-19T01:50:49.556100+0000","last_active":"2024-03-19T01:50:52.991448+0000","last_peered":"2024-03-19T01:50:52.991448+0000","last_clean":"2024-03-19T01:50:52.991448+0000","last_became_active":"2024-03-19T01:50:49.555342+0000","last_became_peered":"2024-03-19T01:50:49.555342+0000","last_unstale":"2024-03-19T01:50:52.991448+0000","last_undegraded":"2024-03-19T01:50:52.991448+0000","last_fullsized":"2024-03-19T01:50:52.991448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:50:15.712384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063335+0000","last_change":"2024-03-19T01:50:50.552792+0000","last_active":"2024-03-19T01:50:52.063335+0000","last_peered":"2024-03-19T01:50:52.063335+0000","last_clean":"2024-03-19T01:50:52.063335+0000","last_became_active":"2024-03-19T01:50:50.552438+0000","last_became_peered":"2024-03-19T01:50:50.552438+0000","last_unstale":"2024-03-19T01:50:52.063335+0000","last_undegraded":"2024-03-19T01:50:52.063335+0000","last_fullsized":"2024-03-19T01:50:52.063335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:42:10.436248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.974847+0000","last_change":"2024-03-19T01:50:49.546948+0000","last_active":"2024-03-19T01:50:55.974847+0000","last_peered":"2024-03-19T01:50:55.974847+0000","last_clean":"2024-03-19T01:50:55.974847+0000","last_became_active":"2024-03-19T01:50:49.546610+0000","last_became_peered":"2024-03-19T01:50:49.546610+0000","last_unstale":"2024-03-19T01:50:55.974847+0000","last_undegraded":"2024-03-19T01:50:55.974847+0000","last_fullsized":"2024-03-19T01:50:55.974847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:06:09.973827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062889+0000","last_change":"2024-03-19T01:50:50.552703+0000","last_active":"2024-03-19T01:50:52.062889+0000","last_peered":"2024-03-19T01:50:52.062889+0000","last_clean":"2024-03-19T01:50:52.062889+0000","last_became_active":"2024-03-19T01:50:50.552337+0000","last_became_peered":"2024-03-19T01:50:50.552337+0000","last_unstale":"2024-03-19T01:50:52.062889+0000","last_undegraded":"2024-03-19T01:50:52.062889+0000","last_fullsized":"2024-03-19T01:50:52.062889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:27.912513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062973+0000","last_change":"2024-03-19T01:50:49.554531+0000","last_active":"2024-03-19T01:50:52.062973+0000","last_peered":"2024-03-19T01:50:52.062973+0000","last_clean":"2024-03-19T01:50:52.062973+0000","last_became_active":"2024-03-19T01:50:49.554196+0000","last_became_peered":"2024-03-19T01:50:49.554196+0000","last_unstale":"2024-03-19T01:50:52.062973+0000","last_undegraded":"2024-03-19T01:50:52.062973+0000","last_fullsized":"2024-03-19T01:50:52.062973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:04.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062973+0000","last_change":"2024-03-19T01:50:50.558390+0000","last_active":"2024-03-19T01:50:52.062973+0000","last_peered":"2024-03-19T01:50:52.062973+0000","last_clean":"2024-03-19T01:50:52.062973+0000","last_became_active":"2024-03-19T01:50:50.558022+0000","last_became_peered":"2024-03-19T01:50:50.558022+0000","last_unstale":"2024-03-19T01:50:52.062973+0000","last_undegraded":"2024-03-19T01:50:52.062973+0000","last_fullsized":"2024-03-19T01:50:52.062973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:20.794803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549973+0000","last_change":"2024-03-19T01:50:49.553203+0000","last_active":"2024-03-19T01:50:51.549973+0000","last_peered":"2024-03-19T01:50:51.549973+0000","last_clean":"2024-03-19T01:50:51.549973+0000","last_became_active":"2024-03-19T01:50:49.552243+0000","last_became_peered":"2024-03-19T01:50:49.552243+0000","last_unstale":"2024-03-19T01:50:51.549973+0000","last_undegraded":"2024-03-19T01:50:51.549973+0000","last_fullsized":"2024-03-19T01:50:51.549973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:16.313075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063066+0000","last_change":"2024-03-19T01:50:50.567317+0000","last_active":"2024-03-19T01:50:52.063066+0000","last_peered":"2024-03-19T01:50:52.063066+0000","last_clean":"2024-03-19T01:50:52.063066+0000","last_became_active":"2024-03-19T01:50:50.566744+0000","last_became_peered":"2024-03-19T01:50:50.566744+0000","last_unstale":"2024-03-19T01:50:52.063066+0000","last_undegraded":"2024-03-19T01:50:52.063066+0000","last_fullsized":"2024-03-19T01:50:52.063066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:01:34.239148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062826+0000","last_change":"2024-03-19T01:50:50.571361+0000","last_active":"2024-03-19T01:50:52.062826+0000","last_peered":"2024-03-19T01:50:52.062826+0000","last_clean":"2024-03-19T01:50:52.062826+0000","last_became_active":"2024-03-19T01:50:50.570845+0000","last_became_peered":"2024-03-19T01:50:50.570845+0000","last_unstale":"2024-03-19T01:50:52.062826+0000","last_undegraded":"2024-03-19T01:50:52.062826+0000","last_fullsized":"2024-03-19T01:50:52.062826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:27:46.816815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549658+0000","last_change":"2024-03-19T01:50:50.564337+0000","last_active":"2024-03-19T01:50:51.549658+0000","last_peered":"2024-03-19T01:50:51.549658+0000","last_clean":"2024-03-19T01:50:51.549658+0000","last_became_active":"2024-03-19T01:50:50.564002+0000","last_became_peered":"2024-03-19T01:50:50.564002+0000","last_unstale":"2024-03-19T01:50:51.549658+0000","last_undegraded":"2024-03-19T01:50:51.549658+0000","last_fullsized":"2024-03-19T01:50:51.549658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:19:51.072829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549914+0000","last_change":"2024-03-19T01:50:50.560962+0000","last_active":"2024-03-19T01:50:51.549914+0000","last_peered":"2024-03-19T01:50:51.549914+0000","last_clean":"2024-03-19T01:50:51.549914+0000","last_became_active":"2024-03-19T01:50:50.560427+0000","last_became_peered":"2024-03-19T01:50:50.560427+0000","last_unstale":"2024-03-19T01:50:51.549914+0000","last_undegraded":"2024-03-19T01:50:51.549914+0000","last_fullsized":"2024-03-19T01:50:51.549914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:54.472200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062303+0000","last_change":"2024-03-19T01:50:50.557806+0000","last_active":"2024-03-19T01:50:52.062303+0000","last_peered":"2024-03-19T01:50:52.062303+0000","last_clean":"2024-03-19T01:50:52.062303+0000","last_became_active":"2024-03-19T01:50:50.557494+0000","last_became_peered":"2024-03-19T01:50:50.557494+0000","last_unstale":"2024-03-19T01:50:52.062303+0000","last_undegraded":"2024-03-19T01:50:52.062303+0000","last_fullsized":"2024-03-19T01:50:52.062303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:35:53.787047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550340+0000","last_change":"2024-03-19T01:50:50.554414+0000","last_active":"2024-03-19T01:50:51.550340+0000","last_peered":"2024-03-19T01:50:51.550340+0000","last_clean":"2024-03-19T01:50:51.550340+0000","last_became_active":"2024-03-19T01:50:50.554010+0000","last_became_peered":"2024-03-19T01:50:50.554010+0000","last_unstale":"2024-03-19T01:50:51.550340+0000","last_undegraded":"2024-03-19T01:50:51.550340+0000","last_fullsized":"2024-03-19T01:50:51.550340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:02:54.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062855+0000","last_change":"2024-03-19T01:50:50.567759+0000","last_active":"2024-03-19T01:50:52.062855+0000","last_peered":"2024-03-19T01:50:52.062855+0000","last_clean":"2024-03-19T01:50:52.062855+0000","last_became_active":"2024-03-19T01:50:50.566529+0000","last_became_peered":"2024-03-19T01:50:50.566529+0000","last_unstale":"2024-03-19T01:50:52.062855+0000","last_undegraded":"2024-03-19T01:50:52.062855+0000","last_fullsized":"2024-03-19T01:50:52.062855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:25:05.002161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550007+0000","last_change":"2024-03-19T01:50:50.563915+0000","last_active":"2024-03-19T01:50:51.550007+0000","last_peered":"2024-03-19T01:50:51.550007+0000","last_clean":"2024-03-19T01:50:51.550007+0000","last_became_active":"2024-03-19T01:50:50.563596+0000","last_became_peered":"2024-03-19T01:50:50.563596+0000","last_unstale":"2024-03-19T01:50:51.550007+0000","last_undegraded":"2024-03-19T01:50:51.550007+0000","last_fullsized":"2024-03-19T01:50:51.550007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:12.832402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063117+0000","last_change":"2024-03-19T01:50:50.556507+0000","last_active":"2024-03-19T01:50:52.063117+0000","last_peered":"2024-03-19T01:50:52.063117+0000","last_clean":"2024-03-19T01:50:52.063117+0000","last_became_active":"2024-03-19T01:50:50.556197+0000","last_became_peered":"2024-03-19T01:50:50.556197+0000","last_unstale":"2024-03-19T01:50:52.063117+0000","last_undegraded":"2024-03-19T01:50:52.063117+0000","last_fullsized":"2024-03-19T01:50:52.063117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:25:26.919274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062737+0000","last_change":"2024-03-19T01:50:50.554403+0000","last_active":"2024-03-19T01:50:52.062737+0000","last_peered":"2024-03-19T01:50:52.062737+0000","last_clean":"2024-03-19T01:50:52.062737+0000","last_became_active":"2024-03-19T01:50:50.553985+0000","last_became_peered":"2024-03-19T01:50:50.553985+0000","last_unstale":"2024-03-19T01:50:52.062737+0000","last_undegraded":"2024-03-19T01:50:52.062737+0000","last_fullsized":"2024-03-19T01:50:52.062737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:50.923915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062685+0000","last_change":"2024-03-19T01:50:50.554391+0000","last_active":"2024-03-19T01:50:52.062685+0000","last_peered":"2024-03-19T01:50:52.062685+0000","last_clean":"2024-03-19T01:50:52.062685+0000","last_became_active":"2024-03-19T01:50:50.552987+0000","last_became_peered":"2024-03-19T01:50:50.552987+0000","last_unstale":"2024-03-19T01:50:52.062685+0000","last_undegraded":"2024-03-19T01:50:52.062685+0000","last_fullsized":"2024-03-19T01:50:52.062685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:07.107613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.551956+0000","last_change":"2024-03-19T01:50:50.556172+0000","last_active":"2024-03-19T01:50:51.551956+0000","last_peered":"2024-03-19T01:50:51.551956+0000","last_clean":"2024-03-19T01:50:51.551956+0000","last_became_active":"2024-03-19T01:50:50.555468+0000","last_became_peered":"2024-03-19T01:50:50.555468+0000","last_unstale":"2024-03-19T01:50:51.551956+0000","last_undegraded":"2024-03-19T01:50:51.551956+0000","last_fullsized":"2024-03-19T01:50:51.551956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:28:03.005962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550166+0000","last_change":"2024-03-19T01:50:50.550915+0000","last_active":"2024-03-19T01:50:51.550166+0000","last_peered":"2024-03-19T01:50:51.550166+0000","last_clean":"2024-03-19T01:50:51.550166+0000","last_became_active":"2024-03-19T01:50:50.550535+0000","last_became_peered":"2024-03-19T01:50:50.550535+0000","last_unstale":"2024-03-19T01:50:51.550166+0000","last_undegraded":"2024-03-19T01:50:51.550166+0000","last_fullsized":"2024-03-19T01:50:51.550166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:05:21.959483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063005+0000","last_change":"2024-03-19T01:50:50.567475+0000","last_active":"2024-03-19T01:50:52.063005+0000","last_peered":"2024-03-19T01:50:52.063005+0000","last_clean":"2024-03-19T01:50:52.063005+0000","last_became_active":"2024-03-19T01:50:50.567103+0000","last_became_peered":"2024-03-19T01:50:50.567103+0000","last_unstale":"2024-03-19T01:50:52.063005+0000","last_undegraded":"2024-03-19T01:50:52.063005+0000","last_fullsized":"2024-03-19T01:50:52.063005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:41.866610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244932,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5222,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":143,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":54049,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":1031457,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":439643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":1030681,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":461779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":438951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1029071,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":40556,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:50:58.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-19T01:50:58.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:58 smithi159 ceph-mon[67110]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:58.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:58 smithi159 ceph-mon[67110]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:58.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:58 smithi159 ceph-mon[67110]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:50:58.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:58 smithi159 ceph-mon[67110]: pgmap v294: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 646 B/s rd, 4.9 KiB/s wr, 18 op/s 2024-03-19T01:50:58.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:58 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2999399044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:50:58.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.536+0000 7fd657c85640 1 -- 172.21.15.92:0/2368331541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd650101930 msgr2=0x7fd650101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.536+0000 7fd657c85640 1 --2- 172.21.15.92:0/2368331541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd650101930 0x7fd650101db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd640009900 tx=0x7fd64002bc30 comp rx=0 tx=0).stop 2024-03-19T01:50:58.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.536+0000 7fd657c85640 1 -- 172.21.15.92:0/2368331541 shutdown_connections 2024-03-19T01:50:58.538 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.536+0000 7fd657c85640 1 --2- 172.21.15.92:0/2368331541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6501022f0 0x7fd650108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.536+0000 7fd657c85640 1 --2- 172.21.15.92:0/2368331541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd650101930 0x7fd650101db0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.536+0000 7fd657c85640 1 --2- 172.21.15.92:0/2368331541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd650100740 0x7fd650100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.536+0000 7fd657c85640 1 -- 172.21.15.92:0/2368331541 >> 172.21.15.92:0/2368331541 conn(0x7fd6500fbcb0 msgr2=0x7fd6500fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.536+0000 7fd657c85640 1 -- 172.21.15.92:0/2368331541 shutdown_connections 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.536+0000 7fd657c85640 1 -- 172.21.15.92:0/2368331541 wait complete. 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd657c85640 1 Processor -- start 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd657c85640 1 -- start start 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd657c85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd650100740 0x7fd650197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd657c85640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650101930 0x7fd6501980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd657c85640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6501022f0 0x7fd65019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd657c85640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd65010c3a0 con 0x7fd6501022f0 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd657c85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd65010c220 con 0x7fd650100740 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd657c85640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd65010c520 con 0x7fd650101930 2024-03-19T01:50:58.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd656222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6501022f0 0x7fd65019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd656222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6501022f0 0x7fd65019f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37944/0 (socket says 172.21.15.92:37944) 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd656222640 1 -- 172.21.15.92:0/1220484992 learned_addr learned my addr 172.21.15.92:0/1220484992 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd655220640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650101930 0x7fd6501980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd655a21640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd650100740 0x7fd650197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd656222640 1 -- 172.21.15.92:0/1220484992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650101930 msgr2=0x7fd6501980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd656222640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650101930 0x7fd6501980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd656222640 1 -- 172.21.15.92:0/1220484992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd650100740 msgr2=0x7fd650197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd656222640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd650100740 0x7fd650197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd656222640 1 -- 172.21.15.92:0/1220484992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65019f770 con 0x7fd6501022f0 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.537+0000 7fd656222640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6501022f0 0x7fd65019f100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd64800eb10 tx=0x7fd64800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.538+0000 7fd646ffd640 1 -- 172.21.15.92:0/1220484992 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648004b20 con 0x7fd6501022f0 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.538+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65019fa60 con 0x7fd6501022f0 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.538+0000 7fd646ffd640 1 -- 172.21.15.92:0/1220484992 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd648004cc0 con 0x7fd6501022f0 2024-03-19T01:50:58.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.538+0000 7fd646ffd640 1 -- 172.21.15.92:0/1220484992 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648010760 con 0x7fd6501022f0 2024-03-19T01:50:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.538+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65019ff80 con 0x7fd6501022f0 2024-03-19T01:50:58.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.539+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6501a01b0 con 0x7fd6501022f0 2024-03-19T01:50:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.540+0000 7fd646ffd640 1 -- 172.21.15.92:0/1220484992 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd648005020 con 0x7fd6501022f0 2024-03-19T01:50:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.540+0000 7fd646ffd640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd63c0a2890 0x7fd63c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:50:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.540+0000 7fd655a21640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd63c0a2890 0x7fd63c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:50:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.541+0000 7fd646ffd640 1 -- 172.21.15.92:0/1220484992 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd6480cefb0 con 0x7fd6501022f0 2024-03-19T01:50:58.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.541+0000 7fd655a21640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd63c0a2890 0x7fd63c0a4d50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd650101790 tx=0x7fd64c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:50:58.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.544+0000 7fd646ffd640 1 -- 172.21.15.92:0/1220484992 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd64809c0a0 con 0x7fd6501022f0 2024-03-19T01:50:58.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.777+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6500fde70 con 0x7fd63c0a2890 2024-03-19T01:50:58.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.784+0000 7fd646ffd640 1 -- 172.21.15.92:0/1220484992 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258943 (secure 0 0 0) 0x7fd6500fde70 con 0x7fd63c0a2890 2024-03-19T01:50:58.787 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:50:58.788 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd63c0a2890 msgr2=0x7fd63c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd63c0a2890 0x7fd63c0a4d50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd650101790 tx=0x7fd64c005f90 comp rx=0 tx=0).stop 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6501022f0 msgr2=0x7fd65019f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6501022f0 0x7fd65019f100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd64800eb10 tx=0x7fd64800c2d0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 shutdown_connections 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6501022f0 0x7fd65019f100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd63c0a2890 0x7fd63c0a4d50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650101930 0x7fd6501980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 --2- 172.21.15.92:0/1220484992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd650100740 0x7fd650197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:50:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 >> 172.21.15.92:0/1220484992 conn(0x7fd6500fbcb0 msgr2=0x7fd6500fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:50:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 shutdown_connections 2024-03-19T01:50:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:50:58.787+0000 7fd657c85640 1 -- 172.21.15.92:0/1220484992 wait complete. 2024-03-19T01:50:58.799 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-03-19T01:50:57.446099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595590,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":133,"num_read_kb":114,"num_write":324,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351964,"kb_used_data":11088,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589860,"statfs":{"total":1151940427776,"available":1151580016640,"internally_reserved":0,"allocated":11354112,"data_stored":7066356,"data_compressed":107082,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":348894524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5222,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":143,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.915711"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063395+0000","last_change":"2024-03-19T01:50:50.561781+0000","last_active":"2024-03-19T01:50:52.063395+0000","last_peered":"2024-03-19T01:50:52.063395+0000","last_clean":"2024-03-19T01:50:52.063395+0000","last_became_active":"2024-03-19T01:50:50.561402+0000","last_became_peered":"2024-03-19T01:50:50.561402+0000","last_unstale":"2024-03-19T01:50:52.063395+0000","last_undegraded":"2024-03-19T01:50:52.063395+0000","last_fullsized":"2024-03-19T01:50:52.063395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:07.257394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062355+0000","last_change":"2024-03-19T01:50:50.558625+0000","last_active":"2024-03-19T01:50:52.062355+0000","last_peered":"2024-03-19T01:50:52.062355+0000","last_clean":"2024-03-19T01:50:52.062355+0000","last_became_active":"2024-03-19T01:50:50.557349+0000","last_became_peered":"2024-03-19T01:50:50.557349+0000","last_unstale":"2024-03-19T01:50:52.062355+0000","last_undegraded":"2024-03-19T01:50:52.062355+0000","last_fullsized":"2024-03-19T01:50:52.062355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:05.791529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062322+0000","last_change":"2024-03-19T01:50:50.554103+0000","last_active":"2024-03-19T01:50:52.062322+0000","last_peered":"2024-03-19T01:50:52.062322+0000","last_clean":"2024-03-19T01:50:52.062322+0000","last_became_active":"2024-03-19T01:50:50.553734+0000","last_became_peered":"2024-03-19T01:50:50.553734+0000","last_unstale":"2024-03-19T01:50:52.062322+0000","last_undegraded":"2024-03-19T01:50:52.062322+0000","last_fullsized":"2024-03-19T01:50:52.062322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:18:02.768774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.697597+0000","last_change":"2024-03-19T01:50:50.571980+0000","last_active":"2024-03-19T01:50:51.697597+0000","last_peered":"2024-03-19T01:50:51.697597+0000","last_clean":"2024-03-19T01:50:51.697597+0000","last_became_active":"2024-03-19T01:50:50.571679+0000","last_became_peered":"2024-03-19T01:50:50.571679+0000","last_unstale":"2024-03-19T01:50:51.697597+0000","last_undegraded":"2024-03-19T01:50:51.697597+0000","last_fullsized":"2024-03-19T01:50:51.697597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:30:08.946509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062334+0000","last_change":"2024-03-19T01:50:50.563314+0000","last_active":"2024-03-19T01:50:52.062334+0000","last_peered":"2024-03-19T01:50:52.062334+0000","last_clean":"2024-03-19T01:50:52.062334+0000","last_became_active":"2024-03-19T01:50:50.562953+0000","last_became_peered":"2024-03-19T01:50:50.562953+0000","last_unstale":"2024-03-19T01:50:52.062334+0000","last_undegraded":"2024-03-19T01:50:52.062334+0000","last_fullsized":"2024-03-19T01:50:52.062334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:38.139670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063398+0000","last_change":"2024-03-19T01:50:50.555305+0000","last_active":"2024-03-19T01:50:52.063398+0000","last_peered":"2024-03-19T01:50:52.063398+0000","last_clean":"2024-03-19T01:50:52.063398+0000","last_became_active":"2024-03-19T01:50:50.554930+0000","last_became_peered":"2024-03-19T01:50:50.554930+0000","last_unstale":"2024-03-19T01:50:52.063398+0000","last_undegraded":"2024-03-19T01:50:52.063398+0000","last_fullsized":"2024-03-19T01:50:52.063398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:05:03.418572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062714+0000","last_change":"2024-03-19T01:50:50.557381+0000","last_active":"2024-03-19T01:50:52.062714+0000","last_peered":"2024-03-19T01:50:52.062714+0000","last_clean":"2024-03-19T01:50:52.062714+0000","last_became_active":"2024-03-19T01:50:50.556804+0000","last_became_peered":"2024-03-19T01:50:50.556804+0000","last_unstale":"2024-03-19T01:50:52.062714+0000","last_undegraded":"2024-03-19T01:50:52.062714+0000","last_fullsized":"2024-03-19T01:50:52.062714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:54:00.676606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062622+0000","last_change":"2024-03-19T01:50:50.547526+0000","last_active":"2024-03-19T01:50:52.062622+0000","last_peered":"2024-03-19T01:50:52.062622+0000","last_clean":"2024-03-19T01:50:52.062622+0000","last_became_active":"2024-03-19T01:50:50.547053+0000","last_became_peered":"2024-03-19T01:50:50.547053+0000","last_unstale":"2024-03-19T01:50:52.062622+0000","last_undegraded":"2024-03-19T01:50:52.062622+0000","last_fullsized":"2024-03-19T01:50:52.062622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:40:17.050026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.552369+0000","last_change":"2024-03-19T01:50:50.562970+0000","last_active":"2024-03-19T01:50:51.552369+0000","last_peered":"2024-03-19T01:50:51.552369+0000","last_clean":"2024-03-19T01:50:51.552369+0000","last_became_active":"2024-03-19T01:50:50.562372+0000","last_became_peered":"2024-03-19T01:50:50.562372+0000","last_unstale":"2024-03-19T01:50:51.552369+0000","last_undegraded":"2024-03-19T01:50:51.552369+0000","last_fullsized":"2024-03-19T01:50:51.552369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:57.861083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063799+0000","last_change":"2024-03-19T01:50:50.559369+0000","last_active":"2024-03-19T01:50:52.063799+0000","last_peered":"2024-03-19T01:50:52.063799+0000","last_clean":"2024-03-19T01:50:52.063799+0000","last_became_active":"2024-03-19T01:50:50.559062+0000","last_became_peered":"2024-03-19T01:50:50.559062+0000","last_unstale":"2024-03-19T01:50:52.063799+0000","last_undegraded":"2024-03-19T01:50:52.063799+0000","last_fullsized":"2024-03-19T01:50:52.063799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:08:22.977566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062876+0000","last_change":"2024-03-19T01:50:50.567000+0000","last_active":"2024-03-19T01:50:52.062876+0000","last_peered":"2024-03-19T01:50:52.062876+0000","last_clean":"2024-03-19T01:50:52.062876+0000","last_became_active":"2024-03-19T01:50:50.566613+0000","last_became_peered":"2024-03-19T01:50:50.566613+0000","last_unstale":"2024-03-19T01:50:52.062876+0000","last_undegraded":"2024-03-19T01:50:52.062876+0000","last_fullsized":"2024-03-19T01:50:52.062876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:40:39.132559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.697589+0000","last_change":"2024-03-19T01:50:50.564354+0000","last_active":"2024-03-19T01:50:51.697589+0000","last_peered":"2024-03-19T01:50:51.697589+0000","last_clean":"2024-03-19T01:50:51.697589+0000","last_became_active":"2024-03-19T01:50:50.563997+0000","last_became_peered":"2024-03-19T01:50:50.563997+0000","last_unstale":"2024-03-19T01:50:51.697589+0000","last_undegraded":"2024-03-19T01:50:51.697589+0000","last_fullsized":"2024-03-19T01:50:51.697589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:27:50.266867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550077+0000","last_change":"2024-03-19T01:50:50.539164+0000","last_active":"2024-03-19T01:50:51.550077+0000","last_peered":"2024-03-19T01:50:51.550077+0000","last_clean":"2024-03-19T01:50:51.550077+0000","last_became_active":"2024-03-19T01:50:50.538792+0000","last_became_peered":"2024-03-19T01:50:50.538792+0000","last_unstale":"2024-03-19T01:50:51.550077+0000","last_undegraded":"2024-03-19T01:50:51.550077+0000","last_fullsized":"2024-03-19T01:50:51.550077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:07:56.700994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062274+0000","last_change":"2024-03-19T01:50:50.567590+0000","last_active":"2024-03-19T01:50:52.062274+0000","last_peered":"2024-03-19T01:50:52.062274+0000","last_clean":"2024-03-19T01:50:52.062274+0000","last_became_active":"2024-03-19T01:50:50.567225+0000","last_became_peered":"2024-03-19T01:50:50.567225+0000","last_unstale":"2024-03-19T01:50:52.062274+0000","last_undegraded":"2024-03-19T01:50:52.062274+0000","last_fullsized":"2024-03-19T01:50:52.062274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:55.537567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062115+0000","last_change":"2024-03-19T01:50:50.571815+0000","last_active":"2024-03-19T01:50:52.062115+0000","last_peered":"2024-03-19T01:50:52.062115+0000","last_clean":"2024-03-19T01:50:52.062115+0000","last_became_active":"2024-03-19T01:50:50.571484+0000","last_became_peered":"2024-03-19T01:50:50.571484+0000","last_unstale":"2024-03-19T01:50:52.062115+0000","last_undegraded":"2024-03-19T01:50:52.062115+0000","last_fullsized":"2024-03-19T01:50:52.062115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:40:12.230817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.551949+0000","last_change":"2024-03-19T01:50:50.558616+0000","last_active":"2024-03-19T01:50:51.551949+0000","last_peered":"2024-03-19T01:50:51.551949+0000","last_clean":"2024-03-19T01:50:51.551949+0000","last_became_active":"2024-03-19T01:50:50.558173+0000","last_became_peered":"2024-03-19T01:50:50.558173+0000","last_unstale":"2024-03-19T01:50:51.551949+0000","last_undegraded":"2024-03-19T01:50:51.551949+0000","last_fullsized":"2024-03-19T01:50:51.551949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:04:47.174828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063149+0000","last_change":"2024-03-19T01:50:50.563450+0000","last_active":"2024-03-19T01:50:52.063149+0000","last_peered":"2024-03-19T01:50:52.063149+0000","last_clean":"2024-03-19T01:50:52.063149+0000","last_became_active":"2024-03-19T01:50:50.562663+0000","last_became_peered":"2024-03-19T01:50:50.562663+0000","last_unstale":"2024-03-19T01:50:52.063149+0000","last_undegraded":"2024-03-19T01:50:52.063149+0000","last_fullsized":"2024-03-19T01:50:52.063149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:57:47.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.840669+0000","last_change":"2024-03-19T01:50:50.559848+0000","last_active":"2024-03-19T01:50:51.840669+0000","last_peered":"2024-03-19T01:50:51.840669+0000","last_clean":"2024-03-19T01:50:51.840669+0000","last_became_active":"2024-03-19T01:50:50.559515+0000","last_became_peered":"2024-03-19T01:50:50.559515+0000","last_unstale":"2024-03-19T01:50:51.840669+0000","last_undegraded":"2024-03-19T01:50:51.840669+0000","last_fullsized":"2024-03-19T01:50:51.840669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:16:15.262613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549838+0000","last_change":"2024-03-19T01:50:50.559994+0000","last_active":"2024-03-19T01:50:51.549838+0000","last_peered":"2024-03-19T01:50:51.549838+0000","last_clean":"2024-03-19T01:50:51.549838+0000","last_became_active":"2024-03-19T01:50:50.558759+0000","last_became_peered":"2024-03-19T01:50:50.558759+0000","last_unstale":"2024-03-19T01:50:51.549838+0000","last_undegraded":"2024-03-19T01:50:51.549838+0000","last_fullsized":"2024-03-19T01:50:51.549838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:53.957096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.974634+0000","last_change":"2024-03-19T01:50:49.550130+0000","last_active":"2024-03-19T01:50:55.974634+0000","last_peered":"2024-03-19T01:50:55.974634+0000","last_clean":"2024-03-19T01:50:55.974634+0000","last_became_active":"2024-03-19T01:50:49.549374+0000","last_became_peered":"2024-03-19T01:50:49.549374+0000","last_unstale":"2024-03-19T01:50:55.974634+0000","last_undegraded":"2024-03-19T01:50:55.974634+0000","last_fullsized":"2024-03-19T01:50:55.974634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:14:43.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063109+0000","last_change":"2024-03-19T01:50:50.570319+0000","last_active":"2024-03-19T01:50:52.063109+0000","last_peered":"2024-03-19T01:50:52.063109+0000","last_clean":"2024-03-19T01:50:52.063109+0000","last_became_active":"2024-03-19T01:50:50.569937+0000","last_became_peered":"2024-03-19T01:50:50.569937+0000","last_unstale":"2024-03-19T01:50:52.063109+0000","last_undegraded":"2024-03-19T01:50:52.063109+0000","last_fullsized":"2024-03-19T01:50:52.063109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:38:00.024688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063239+0000","last_change":"2024-03-19T01:50:49.551261+0000","last_active":"2024-03-19T01:50:52.063239+0000","last_peered":"2024-03-19T01:50:52.063239+0000","last_clean":"2024-03-19T01:50:52.063239+0000","last_became_active":"2024-03-19T01:50:49.550949+0000","last_became_peered":"2024-03-19T01:50:49.550949+0000","last_unstale":"2024-03-19T01:50:52.063239+0000","last_undegraded":"2024-03-19T01:50:52.063239+0000","last_fullsized":"2024-03-19T01:50:52.063239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:23:49.064155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062459+0000","last_change":"2024-03-19T01:50:50.554303+0000","last_active":"2024-03-19T01:50:52.062459+0000","last_peered":"2024-03-19T01:50:52.062459+0000","last_clean":"2024-03-19T01:50:52.062459+0000","last_became_active":"2024-03-19T01:50:50.553765+0000","last_became_peered":"2024-03-19T01:50:50.553765+0000","last_unstale":"2024-03-19T01:50:52.062459+0000","last_undegraded":"2024-03-19T01:50:52.062459+0000","last_fullsized":"2024-03-19T01:50:52.062459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:42.246266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.004044+0000","last_change":"2024-03-19T01:50:49.554688+0000","last_active":"2024-03-19T01:50:55.004044+0000","last_peered":"2024-03-19T01:50:55.004044+0000","last_clean":"2024-03-19T01:50:55.004044+0000","last_became_active":"2024-03-19T01:50:49.554289+0000","last_became_peered":"2024-03-19T01:50:49.554289+0000","last_unstale":"2024-03-19T01:50:55.004044+0000","last_undegraded":"2024-03-19T01:50:55.004044+0000","last_fullsized":"2024-03-19T01:50:55.004044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:13.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.551911+0000","last_change":"2024-03-19T01:50:50.563305+0000","last_active":"2024-03-19T01:50:51.551911+0000","last_peered":"2024-03-19T01:50:51.551911+0000","last_clean":"2024-03-19T01:50:51.551911+0000","last_became_active":"2024-03-19T01:50:50.561819+0000","last_became_peered":"2024-03-19T01:50:50.561819+0000","last_unstale":"2024-03-19T01:50:51.551911+0000","last_undegraded":"2024-03-19T01:50:51.551911+0000","last_fullsized":"2024-03-19T01:50:51.551911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:11:56.175649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063344+0000","last_change":"2024-03-19T01:50:49.552522+0000","last_active":"2024-03-19T01:50:52.063344+0000","last_peered":"2024-03-19T01:50:52.063344+0000","last_clean":"2024-03-19T01:50:52.063344+0000","last_became_active":"2024-03-19T01:50:49.552083+0000","last_became_peered":"2024-03-19T01:50:49.552083+0000","last_unstale":"2024-03-19T01:50:52.063344+0000","last_undegraded":"2024-03-19T01:50:52.063344+0000","last_fullsized":"2024-03-19T01:50:52.063344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:51:50.069723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063168+0000","last_change":"2024-03-19T01:50:50.560178+0000","last_active":"2024-03-19T01:50:52.063168+0000","last_peered":"2024-03-19T01:50:52.063168+0000","last_clean":"2024-03-19T01:50:52.063168+0000","last_became_active":"2024-03-19T01:50:50.559853+0000","last_became_peered":"2024-03-19T01:50:50.559853+0000","last_unstale":"2024-03-19T01:50:52.063168+0000","last_undegraded":"2024-03-19T01:50:52.063168+0000","last_fullsized":"2024-03-19T01:50:52.063168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:10:04.812207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.993399+0000","last_change":"2024-03-19T01:50:49.550194+0000","last_active":"2024-03-19T01:50:52.993399+0000","last_peered":"2024-03-19T01:50:52.993399+0000","last_clean":"2024-03-19T01:50:52.993399+0000","last_became_active":"2024-03-19T01:50:49.549801+0000","last_became_peered":"2024-03-19T01:50:49.549801+0000","last_unstale":"2024-03-19T01:50:52.993399+0000","last_undegraded":"2024-03-19T01:50:52.993399+0000","last_fullsized":"2024-03-19T01:50:52.993399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:43.110159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062323+0000","last_change":"2024-03-19T01:50:50.560913+0000","last_active":"2024-03-19T01:50:52.062323+0000","last_peered":"2024-03-19T01:50:52.062323+0000","last_clean":"2024-03-19T01:50:52.062323+0000","last_became_active":"2024-03-19T01:50:50.559210+0000","last_became_peered":"2024-03-19T01:50:50.559210+0000","last_unstale":"2024-03-19T01:50:52.062323+0000","last_undegraded":"2024-03-19T01:50:52.062323+0000","last_fullsized":"2024-03-19T01:50:52.062323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:14.814374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063151+0000","last_change":"2024-03-19T01:50:49.548778+0000","last_active":"2024-03-19T01:50:52.063151+0000","last_peered":"2024-03-19T01:50:52.063151+0000","last_clean":"2024-03-19T01:50:52.063151+0000","last_became_active":"2024-03-19T01:50:49.548414+0000","last_became_peered":"2024-03-19T01:50:49.548414+0000","last_unstale":"2024-03-19T01:50:52.063151+0000","last_undegraded":"2024-03-19T01:50:52.063151+0000","last_fullsized":"2024-03-19T01:50:52.063151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:03:37.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063650+0000","last_change":"2024-03-19T01:50:50.559242+0000","last_active":"2024-03-19T01:50:52.063650+0000","last_peered":"2024-03-19T01:50:52.063650+0000","last_clean":"2024-03-19T01:50:52.063650+0000","last_became_active":"2024-03-19T01:50:50.558904+0000","last_became_peered":"2024-03-19T01:50:50.558904+0000","last_unstale":"2024-03-19T01:50:52.063650+0000","last_undegraded":"2024-03-19T01:50:52.063650+0000","last_fullsized":"2024-03-19T01:50:52.063650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:59.249180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549849+0000","last_change":"2024-03-19T01:50:49.548331+0000","last_active":"2024-03-19T01:50:51.549849+0000","last_peered":"2024-03-19T01:50:51.549849+0000","last_clean":"2024-03-19T01:50:51.549849+0000","last_became_active":"2024-03-19T01:50:49.547993+0000","last_became_peered":"2024-03-19T01:50:49.547993+0000","last_unstale":"2024-03-19T01:50:51.549849+0000","last_undegraded":"2024-03-19T01:50:51.549849+0000","last_fullsized":"2024-03-19T01:50:51.549849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:30.059133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062578+0000","last_change":"2024-03-19T01:50:50.553166+0000","last_active":"2024-03-19T01:50:52.062578+0000","last_peered":"2024-03-19T01:50:52.062578+0000","last_clean":"2024-03-19T01:50:52.062578+0000","last_became_active":"2024-03-19T01:50:50.552520+0000","last_became_peered":"2024-03-19T01:50:50.552520+0000","last_unstale":"2024-03-19T01:50:52.062578+0000","last_undegraded":"2024-03-19T01:50:52.062578+0000","last_fullsized":"2024-03-19T01:50:52.062578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:10:24.515191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991005+0000","last_change":"2024-03-19T01:50:49.548868+0000","last_active":"2024-03-19T01:50:52.991005+0000","last_peered":"2024-03-19T01:50:52.991005+0000","last_clean":"2024-03-19T01:50:52.991005+0000","last_became_active":"2024-03-19T01:50:49.548548+0000","last_became_peered":"2024-03-19T01:50:49.548548+0000","last_unstale":"2024-03-19T01:50:52.991005+0000","last_undegraded":"2024-03-19T01:50:52.991005+0000","last_fullsized":"2024-03-19T01:50:52.991005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:21:36.920067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.840577+0000","last_change":"2024-03-19T01:50:50.559870+0000","last_active":"2024-03-19T01:50:51.840577+0000","last_peered":"2024-03-19T01:50:51.840577+0000","last_clean":"2024-03-19T01:50:51.840577+0000","last_became_active":"2024-03-19T01:50:50.559530+0000","last_became_peered":"2024-03-19T01:50:50.559530+0000","last_unstale":"2024-03-19T01:50:51.840577+0000","last_undegraded":"2024-03-19T01:50:51.840577+0000","last_fullsized":"2024-03-19T01:50:51.840577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:39:48.635416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991521+0000","last_change":"2024-03-19T01:50:49.554733+0000","last_active":"2024-03-19T01:50:52.991521+0000","last_peered":"2024-03-19T01:50:52.991521+0000","last_clean":"2024-03-19T01:50:52.991521+0000","last_became_active":"2024-03-19T01:50:49.554366+0000","last_became_peered":"2024-03-19T01:50:49.554366+0000","last_unstale":"2024-03-19T01:50:52.991521+0000","last_undegraded":"2024-03-19T01:50:52.991521+0000","last_fullsized":"2024-03-19T01:50:52.991521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:53:03.790255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.552222+0000","last_change":"2024-03-19T01:50:50.561526+0000","last_active":"2024-03-19T01:50:51.552222+0000","last_peered":"2024-03-19T01:50:51.552222+0000","last_clean":"2024-03-19T01:50:51.552222+0000","last_became_active":"2024-03-19T01:50:50.561189+0000","last_became_peered":"2024-03-19T01:50:50.561189+0000","last_unstale":"2024-03-19T01:50:51.552222+0000","last_undegraded":"2024-03-19T01:50:51.552222+0000","last_fullsized":"2024-03-19T01:50:51.552222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:15:18.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991239+0000","last_change":"2024-03-19T01:50:49.553701+0000","last_active":"2024-03-19T01:50:52.991239+0000","last_peered":"2024-03-19T01:50:52.991239+0000","last_clean":"2024-03-19T01:50:52.991239+0000","last_became_active":"2024-03-19T01:50:49.553079+0000","last_became_peered":"2024-03-19T01:50:49.553079+0000","last_unstale":"2024-03-19T01:50:52.991239+0000","last_undegraded":"2024-03-19T01:50:52.991239+0000","last_fullsized":"2024-03-19T01:50:52.991239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:19.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062491+0000","last_change":"2024-03-19T01:50:50.565480+0000","last_active":"2024-03-19T01:50:52.062491+0000","last_peered":"2024-03-19T01:50:52.062491+0000","last_clean":"2024-03-19T01:50:52.062491+0000","last_became_active":"2024-03-19T01:50:50.563930+0000","last_became_peered":"2024-03-19T01:50:50.563930+0000","last_unstale":"2024-03-19T01:50:52.062491+0000","last_undegraded":"2024-03-19T01:50:52.062491+0000","last_fullsized":"2024-03-19T01:50:52.062491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:58:55.462003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991869+0000","last_change":"2024-03-19T01:50:49.553573+0000","last_active":"2024-03-19T01:50:52.991869+0000","last_peered":"2024-03-19T01:50:52.991869+0000","last_clean":"2024-03-19T01:50:52.991869+0000","last_became_active":"2024-03-19T01:50:49.553209+0000","last_became_peered":"2024-03-19T01:50:49.553209+0000","last_unstale":"2024-03-19T01:50:52.991869+0000","last_undegraded":"2024-03-19T01:50:52.991869+0000","last_fullsized":"2024-03-19T01:50:52.991869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:47:43.235323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.552095+0000","last_change":"2024-03-19T01:50:50.568133+0000","last_active":"2024-03-19T01:50:51.552095+0000","last_peered":"2024-03-19T01:50:51.552095+0000","last_clean":"2024-03-19T01:50:51.552095+0000","last_became_active":"2024-03-19T01:50:50.567783+0000","last_became_peered":"2024-03-19T01:50:50.567783+0000","last_unstale":"2024-03-19T01:50:51.552095+0000","last_undegraded":"2024-03-19T01:50:51.552095+0000","last_fullsized":"2024-03-19T01:50:51.552095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:16.077111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991348+0000","last_change":"2024-03-19T01:50:49.546463+0000","last_active":"2024-03-19T01:50:52.991348+0000","last_peered":"2024-03-19T01:50:52.991348+0000","last_clean":"2024-03-19T01:50:52.991348+0000","last_became_active":"2024-03-19T01:50:49.546033+0000","last_became_peered":"2024-03-19T01:50:49.546033+0000","last_unstale":"2024-03-19T01:50:52.991348+0000","last_undegraded":"2024-03-19T01:50:52.991348+0000","last_fullsized":"2024-03-19T01:50:52.991348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:29:13.933301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063502+0000","last_change":"2024-03-19T01:50:50.557576+0000","last_active":"2024-03-19T01:50:52.063502+0000","last_peered":"2024-03-19T01:50:52.063502+0000","last_clean":"2024-03-19T01:50:52.063502+0000","last_became_active":"2024-03-19T01:50:50.555024+0000","last_became_peered":"2024-03-19T01:50:50.555024+0000","last_unstale":"2024-03-19T01:50:52.063502+0000","last_undegraded":"2024-03-19T01:50:52.063502+0000","last_fullsized":"2024-03-19T01:50:52.063502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:17.252079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.978230+0000","last_change":"2024-03-19T01:50:49.547209+0000","last_active":"2024-03-19T01:50:55.978230+0000","last_peered":"2024-03-19T01:50:55.978230+0000","last_clean":"2024-03-19T01:50:55.978230+0000","last_became_active":"2024-03-19T01:50:49.546816+0000","last_became_peered":"2024-03-19T01:50:49.546816+0000","last_unstale":"2024-03-19T01:50:55.978230+0000","last_undegraded":"2024-03-19T01:50:55.978230+0000","last_fullsized":"2024-03-19T01:50:55.978230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:46:42.487614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549965+0000","last_change":"2024-03-19T01:50:50.554679+0000","last_active":"2024-03-19T01:50:51.549965+0000","last_peered":"2024-03-19T01:50:51.549965+0000","last_clean":"2024-03-19T01:50:51.549965+0000","last_became_active":"2024-03-19T01:50:50.554361+0000","last_became_peered":"2024-03-19T01:50:50.554361+0000","last_unstale":"2024-03-19T01:50:51.549965+0000","last_undegraded":"2024-03-19T01:50:51.549965+0000","last_fullsized":"2024-03-19T01:50:51.549965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:44:53.385907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991739+0000","last_change":"2024-03-19T01:50:49.547590+0000","last_active":"2024-03-19T01:50:52.991739+0000","last_peered":"2024-03-19T01:50:52.991739+0000","last_clean":"2024-03-19T01:50:52.991739+0000","last_became_active":"2024-03-19T01:50:49.547261+0000","last_became_peered":"2024-03-19T01:50:49.547261+0000","last_unstale":"2024-03-19T01:50:52.991739+0000","last_undegraded":"2024-03-19T01:50:52.991739+0000","last_fullsized":"2024-03-19T01:50:52.991739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:47.353161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063173+0000","last_change":"2024-03-19T01:50:50.554830+0000","last_active":"2024-03-19T01:50:52.063173+0000","last_peered":"2024-03-19T01:50:52.063173+0000","last_clean":"2024-03-19T01:50:52.063173+0000","last_became_active":"2024-03-19T01:50:50.554495+0000","last_became_peered":"2024-03-19T01:50:50.554495+0000","last_unstale":"2024-03-19T01:50:52.063173+0000","last_undegraded":"2024-03-19T01:50:52.063173+0000","last_fullsized":"2024-03-19T01:50:52.063173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:44.098279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.697176+0000","last_change":"2024-03-19T01:50:49.550056+0000","last_active":"2024-03-19T01:50:51.697176+0000","last_peered":"2024-03-19T01:50:51.697176+0000","last_clean":"2024-03-19T01:50:51.697176+0000","last_became_active":"2024-03-19T01:50:49.549404+0000","last_became_peered":"2024-03-19T01:50:49.549404+0000","last_unstale":"2024-03-19T01:50:51.697176+0000","last_undegraded":"2024-03-19T01:50:51.697176+0000","last_fullsized":"2024-03-19T01:50:51.697176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:38:38.656100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062545+0000","last_change":"2024-03-19T01:50:50.559443+0000","last_active":"2024-03-19T01:50:52.062545+0000","last_peered":"2024-03-19T01:50:52.062545+0000","last_clean":"2024-03-19T01:50:52.062545+0000","last_became_active":"2024-03-19T01:50:50.558992+0000","last_became_peered":"2024-03-19T01:50:50.558992+0000","last_unstale":"2024-03-19T01:50:52.062545+0000","last_undegraded":"2024-03-19T01:50:52.062545+0000","last_fullsized":"2024-03-19T01:50:52.062545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:06:52.144816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:53.998770+0000","last_change":"2024-03-19T01:50:49.548423+0000","last_active":"2024-03-19T01:50:53.998770+0000","last_peered":"2024-03-19T01:50:53.998770+0000","last_clean":"2024-03-19T01:50:53.998770+0000","last_became_active":"2024-03-19T01:50:49.548071+0000","last_became_peered":"2024-03-19T01:50:49.548071+0000","last_unstale":"2024-03-19T01:50:53.998770+0000","last_undegraded":"2024-03-19T01:50:53.998770+0000","last_fullsized":"2024-03-19T01:50:53.998770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:06:00.789358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062464+0000","last_change":"2024-03-19T01:50:50.558569+0000","last_active":"2024-03-19T01:50:52.062464+0000","last_peered":"2024-03-19T01:50:52.062464+0000","last_clean":"2024-03-19T01:50:52.062464+0000","last_became_active":"2024-03-19T01:50:50.557991+0000","last_became_peered":"2024-03-19T01:50:50.557991+0000","last_unstale":"2024-03-19T01:50:52.062464+0000","last_undegraded":"2024-03-19T01:50:52.062464+0000","last_fullsized":"2024-03-19T01:50:52.062464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:55:07.199605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063368+0000","last_change":"2024-03-19T01:50:49.552491+0000","last_active":"2024-03-19T01:50:52.063368+0000","last_peered":"2024-03-19T01:50:52.063368+0000","last_clean":"2024-03-19T01:50:52.063368+0000","last_became_active":"2024-03-19T01:50:49.552133+0000","last_became_peered":"2024-03-19T01:50:49.552133+0000","last_unstale":"2024-03-19T01:50:52.063368+0000","last_undegraded":"2024-03-19T01:50:52.063368+0000","last_fullsized":"2024-03-19T01:50:52.063368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:44.458968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063310+0000","last_change":"2024-03-19T01:50:50.566765+0000","last_active":"2024-03-19T01:50:52.063310+0000","last_peered":"2024-03-19T01:50:52.063310+0000","last_clean":"2024-03-19T01:50:52.063310+0000","last_became_active":"2024-03-19T01:50:50.566412+0000","last_became_peered":"2024-03-19T01:50:50.566412+0000","last_unstale":"2024-03-19T01:50:52.063310+0000","last_undegraded":"2024-03-19T01:50:52.063310+0000","last_fullsized":"2024-03-19T01:50:52.063310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:11:54.418614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.840626+0000","last_change":"2024-03-19T01:50:49.549586+0000","last_active":"2024-03-19T01:50:51.840626+0000","last_peered":"2024-03-19T01:50:51.840626+0000","last_clean":"2024-03-19T01:50:51.840626+0000","last_became_active":"2024-03-19T01:50:49.549224+0000","last_became_peered":"2024-03-19T01:50:49.549224+0000","last_unstale":"2024-03-19T01:50:51.840626+0000","last_undegraded":"2024-03-19T01:50:51.840626+0000","last_fullsized":"2024-03-19T01:50:51.840626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:24:30.591771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063627+0000","last_change":"2024-03-19T01:50:50.554120+0000","last_active":"2024-03-19T01:50:52.063627+0000","last_peered":"2024-03-19T01:50:52.063627+0000","last_clean":"2024-03-19T01:50:52.063627+0000","last_became_active":"2024-03-19T01:50:50.553636+0000","last_became_peered":"2024-03-19T01:50:50.553636+0000","last_unstale":"2024-03-19T01:50:52.063627+0000","last_undegraded":"2024-03-19T01:50:52.063627+0000","last_fullsized":"2024-03-19T01:50:52.063627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:27.634688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063446+0000","last_change":"2024-03-19T01:50:49.551532+0000","last_active":"2024-03-19T01:50:52.063446+0000","last_peered":"2024-03-19T01:50:52.063446+0000","last_clean":"2024-03-19T01:50:52.063446+0000","last_became_active":"2024-03-19T01:50:49.551060+0000","last_became_peered":"2024-03-19T01:50:49.551060+0000","last_unstale":"2024-03-19T01:50:52.063446+0000","last_undegraded":"2024-03-19T01:50:52.063446+0000","last_fullsized":"2024-03-19T01:50:52.063446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:00.339785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063563+0000","last_change":"2024-03-19T01:50:50.556606+0000","last_active":"2024-03-19T01:50:52.063563+0000","last_peered":"2024-03-19T01:50:52.063563+0000","last_clean":"2024-03-19T01:50:52.063563+0000","last_became_active":"2024-03-19T01:50:50.556089+0000","last_became_peered":"2024-03-19T01:50:50.556089+0000","last_unstale":"2024-03-19T01:50:52.063563+0000","last_undegraded":"2024-03-19T01:50:52.063563+0000","last_fullsized":"2024-03-19T01:50:52.063563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:24.256326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062902+0000","last_change":"2024-03-19T01:50:50.564926+0000","last_active":"2024-03-19T01:50:52.062902+0000","last_peered":"2024-03-19T01:50:52.062902+0000","last_clean":"2024-03-19T01:50:52.062902+0000","last_became_active":"2024-03-19T01:50:50.564612+0000","last_became_peered":"2024-03-19T01:50:50.564612+0000","last_unstale":"2024-03-19T01:50:52.062902+0000","last_undegraded":"2024-03-19T01:50:52.062902+0000","last_fullsized":"2024-03-19T01:50:52.062902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:42:29.159512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549788+0000","last_change":"2024-03-19T01:50:49.553780+0000","last_active":"2024-03-19T01:50:51.549788+0000","last_peered":"2024-03-19T01:50:51.549788+0000","last_clean":"2024-03-19T01:50:51.549788+0000","last_became_active":"2024-03-19T01:50:49.552795+0000","last_became_peered":"2024-03-19T01:50:49.552795+0000","last_unstale":"2024-03-19T01:50:51.549788+0000","last_undegraded":"2024-03-19T01:50:51.549788+0000","last_fullsized":"2024-03-19T01:50:51.549788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:01:37.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063059+0000","last_change":"2024-03-19T01:50:50.556943+0000","last_active":"2024-03-19T01:50:52.063059+0000","last_peered":"2024-03-19T01:50:52.063059+0000","last_clean":"2024-03-19T01:50:52.063059+0000","last_became_active":"2024-03-19T01:50:50.556646+0000","last_became_peered":"2024-03-19T01:50:50.556646+0000","last_unstale":"2024-03-19T01:50:52.063059+0000","last_undegraded":"2024-03-19T01:50:52.063059+0000","last_fullsized":"2024-03-19T01:50:52.063059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:59:28.712332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991963+0000","last_change":"2024-03-19T01:50:49.549760+0000","last_active":"2024-03-19T01:50:52.991963+0000","last_peered":"2024-03-19T01:50:52.991963+0000","last_clean":"2024-03-19T01:50:52.991963+0000","last_became_active":"2024-03-19T01:50:49.549218+0000","last_became_peered":"2024-03-19T01:50:49.549218+0000","last_unstale":"2024-03-19T01:50:52.991963+0000","last_undegraded":"2024-03-19T01:50:52.991963+0000","last_fullsized":"2024-03-19T01:50:52.991963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:30:00.802902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062980+0000","last_change":"2024-03-19T01:49:36.469079+0000","last_active":"2024-03-19T01:50:52.062980+0000","last_peered":"2024-03-19T01:50:52.062980+0000","last_clean":"2024-03-19T01:50:52.062980+0000","last_became_active":"2024-03-19T01:49:36.468624+0000","last_became_peered":"2024-03-19T01:49:36.468624+0000","last_unstale":"2024-03-19T01:50:52.062980+0000","last_undegraded":"2024-03-19T01:50:52.062980+0000","last_fullsized":"2024-03-19T01:50:52.062980+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:46:07.695833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:46:07.695833+0000","last_clean_scrub_stamp":"2024-03-19T01:46:07.695833+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:06:31.554853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549872+0000","last_change":"2024-03-19T01:50:50.550851+0000","last_active":"2024-03-19T01:50:51.549872+0000","last_peered":"2024-03-19T01:50:51.549872+0000","last_clean":"2024-03-19T01:50:51.549872+0000","last_became_active":"2024-03-19T01:50:50.550581+0000","last_became_peered":"2024-03-19T01:50:50.550581+0000","last_unstale":"2024-03-19T01:50:51.549872+0000","last_undegraded":"2024-03-19T01:50:51.549872+0000","last_fullsized":"2024-03-19T01:50:51.549872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:24:24.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:53.995865+0000","last_change":"2024-03-19T01:50:49.554043+0000","last_active":"2024-03-19T01:50:53.995865+0000","last_peered":"2024-03-19T01:50:53.995865+0000","last_clean":"2024-03-19T01:50:53.995865+0000","last_became_active":"2024-03-19T01:50:49.553597+0000","last_became_peered":"2024-03-19T01:50:49.553597+0000","last_unstale":"2024-03-19T01:50:53.995865+0000","last_undegraded":"2024-03-19T01:50:53.995865+0000","last_fullsized":"2024-03-19T01:50:53.995865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:41:40.545382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.004210+0000","last_change":"2024-03-19T01:50:49.553740+0000","last_active":"2024-03-19T01:50:55.004210+0000","last_peered":"2024-03-19T01:50:55.004210+0000","last_clean":"2024-03-19T01:50:55.004210+0000","last_became_active":"2024-03-19T01:50:49.553230+0000","last_became_peered":"2024-03-19T01:50:49.553230+0000","last_unstale":"2024-03-19T01:50:55.004210+0000","last_undegraded":"2024-03-19T01:50:55.004210+0000","last_fullsized":"2024-03-19T01:50:55.004210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:46:36.633880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550257+0000","last_change":"2024-03-19T01:50:50.554478+0000","last_active":"2024-03-19T01:50:51.550257+0000","last_peered":"2024-03-19T01:50:51.550257+0000","last_clean":"2024-03-19T01:50:51.550257+0000","last_became_active":"2024-03-19T01:50:50.551170+0000","last_became_peered":"2024-03-19T01:50:50.551170+0000","last_unstale":"2024-03-19T01:50:51.550257+0000","last_undegraded":"2024-03-19T01:50:51.550257+0000","last_fullsized":"2024-03-19T01:50:51.550257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:08:12.382239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.005028+0000","last_change":"2024-03-19T01:50:49.554018+0000","last_active":"2024-03-19T01:50:55.005028+0000","last_peered":"2024-03-19T01:50:55.005028+0000","last_clean":"2024-03-19T01:50:55.005028+0000","last_became_active":"2024-03-19T01:50:49.553702+0000","last_became_peered":"2024-03-19T01:50:49.553702+0000","last_unstale":"2024-03-19T01:50:55.005028+0000","last_undegraded":"2024-03-19T01:50:55.005028+0000","last_fullsized":"2024-03-19T01:50:55.005028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:55.051901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063113+0000","last_change":"2024-03-19T01:50:50.565955+0000","last_active":"2024-03-19T01:50:52.063113+0000","last_peered":"2024-03-19T01:50:52.063113+0000","last_clean":"2024-03-19T01:50:52.063113+0000","last_became_active":"2024-03-19T01:50:50.565643+0000","last_became_peered":"2024-03-19T01:50:50.565643+0000","last_unstale":"2024-03-19T01:50:52.063113+0000","last_undegraded":"2024-03-19T01:50:52.063113+0000","last_fullsized":"2024-03-19T01:50:52.063113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:37:42.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.990133+0000","last_change":"2024-03-19T01:50:49.548298+0000","last_active":"2024-03-19T01:50:52.990133+0000","last_peered":"2024-03-19T01:50:52.990133+0000","last_clean":"2024-03-19T01:50:52.990133+0000","last_became_active":"2024-03-19T01:50:49.547941+0000","last_became_peered":"2024-03-19T01:50:49.547941+0000","last_unstale":"2024-03-19T01:50:52.990133+0000","last_undegraded":"2024-03-19T01:50:52.990133+0000","last_fullsized":"2024-03-19T01:50:52.990133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:23:59.221152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062359+0000","last_change":"2024-03-19T01:50:50.563923+0000","last_active":"2024-03-19T01:50:52.062359+0000","last_peered":"2024-03-19T01:50:52.062359+0000","last_clean":"2024-03-19T01:50:52.062359+0000","last_became_active":"2024-03-19T01:50:50.563295+0000","last_became_peered":"2024-03-19T01:50:50.563295+0000","last_unstale":"2024-03-19T01:50:52.062359+0000","last_undegraded":"2024-03-19T01:50:52.062359+0000","last_fullsized":"2024-03-19T01:50:52.062359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:09:47.089720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.990873+0000","last_change":"2024-03-19T01:50:49.548014+0000","last_active":"2024-03-19T01:50:52.990873+0000","last_peered":"2024-03-19T01:50:52.990873+0000","last_clean":"2024-03-19T01:50:52.990873+0000","last_became_active":"2024-03-19T01:50:49.547588+0000","last_became_peered":"2024-03-19T01:50:49.547588+0000","last_unstale":"2024-03-19T01:50:52.990873+0000","last_undegraded":"2024-03-19T01:50:52.990873+0000","last_fullsized":"2024-03-19T01:50:52.990873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:08.258554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062185+0000","last_change":"2024-03-19T01:50:50.562437+0000","last_active":"2024-03-19T01:50:52.062185+0000","last_peered":"2024-03-19T01:50:52.062185+0000","last_clean":"2024-03-19T01:50:52.062185+0000","last_became_active":"2024-03-19T01:50:50.562113+0000","last_became_peered":"2024-03-19T01:50:50.562113+0000","last_unstale":"2024-03-19T01:50:52.062185+0000","last_undegraded":"2024-03-19T01:50:52.062185+0000","last_fullsized":"2024-03-19T01:50:52.062185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:19:31.932037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.976249+0000","last_change":"2024-03-19T01:50:49.547190+0000","last_active":"2024-03-19T01:50:55.976249+0000","last_peered":"2024-03-19T01:50:55.976249+0000","last_clean":"2024-03-19T01:50:55.976249+0000","last_became_active":"2024-03-19T01:50:49.546698+0000","last_became_peered":"2024-03-19T01:50:49.546698+0000","last_unstale":"2024-03-19T01:50:55.976249+0000","last_undegraded":"2024-03-19T01:50:55.976249+0000","last_fullsized":"2024-03-19T01:50:55.976249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:09.557035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062919+0000","last_change":"2024-03-19T01:50:50.571159+0000","last_active":"2024-03-19T01:50:52.062919+0000","last_peered":"2024-03-19T01:50:52.062919+0000","last_clean":"2024-03-19T01:50:52.062919+0000","last_became_active":"2024-03-19T01:50:50.570312+0000","last_became_peered":"2024-03-19T01:50:50.570312+0000","last_unstale":"2024-03-19T01:50:52.062919+0000","last_undegraded":"2024-03-19T01:50:52.062919+0000","last_fullsized":"2024-03-19T01:50:52.062919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:23:16.820114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.003523+0000","last_change":"2024-03-19T01:50:49.550557+0000","last_active":"2024-03-19T01:50:55.003523+0000","last_peered":"2024-03-19T01:50:55.003523+0000","last_clean":"2024-03-19T01:50:55.003523+0000","last_became_active":"2024-03-19T01:50:49.549789+0000","last_became_peered":"2024-03-19T01:50:49.549789+0000","last_unstale":"2024-03-19T01:50:55.003523+0000","last_undegraded":"2024-03-19T01:50:55.003523+0000","last_fullsized":"2024-03-19T01:50:55.003523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:06:14.292689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063001+0000","last_change":"2024-03-19T01:50:50.560116+0000","last_active":"2024-03-19T01:50:52.063001+0000","last_peered":"2024-03-19T01:50:52.063001+0000","last_clean":"2024-03-19T01:50:52.063001+0000","last_became_active":"2024-03-19T01:50:50.559809+0000","last_became_peered":"2024-03-19T01:50:50.559809+0000","last_unstale":"2024-03-19T01:50:52.063001+0000","last_undegraded":"2024-03-19T01:50:52.063001+0000","last_fullsized":"2024-03-19T01:50:52.063001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:26.854262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.991448+0000","last_change":"2024-03-19T01:50:49.556100+0000","last_active":"2024-03-19T01:50:52.991448+0000","last_peered":"2024-03-19T01:50:52.991448+0000","last_clean":"2024-03-19T01:50:52.991448+0000","last_became_active":"2024-03-19T01:50:49.555342+0000","last_became_peered":"2024-03-19T01:50:49.555342+0000","last_unstale":"2024-03-19T01:50:52.991448+0000","last_undegraded":"2024-03-19T01:50:52.991448+0000","last_fullsized":"2024-03-19T01:50:52.991448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:50:15.712384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063335+0000","last_change":"2024-03-19T01:50:50.552792+0000","last_active":"2024-03-19T01:50:52.063335+0000","last_peered":"2024-03-19T01:50:52.063335+0000","last_clean":"2024-03-19T01:50:52.063335+0000","last_became_active":"2024-03-19T01:50:50.552438+0000","last_became_peered":"2024-03-19T01:50:50.552438+0000","last_unstale":"2024-03-19T01:50:52.063335+0000","last_undegraded":"2024-03-19T01:50:52.063335+0000","last_fullsized":"2024-03-19T01:50:52.063335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:42:10.436248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:55.974847+0000","last_change":"2024-03-19T01:50:49.546948+0000","last_active":"2024-03-19T01:50:55.974847+0000","last_peered":"2024-03-19T01:50:55.974847+0000","last_clean":"2024-03-19T01:50:55.974847+0000","last_became_active":"2024-03-19T01:50:49.546610+0000","last_became_peered":"2024-03-19T01:50:49.546610+0000","last_unstale":"2024-03-19T01:50:55.974847+0000","last_undegraded":"2024-03-19T01:50:55.974847+0000","last_fullsized":"2024-03-19T01:50:55.974847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:06:09.973827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062889+0000","last_change":"2024-03-19T01:50:50.552703+0000","last_active":"2024-03-19T01:50:52.062889+0000","last_peered":"2024-03-19T01:50:52.062889+0000","last_clean":"2024-03-19T01:50:52.062889+0000","last_became_active":"2024-03-19T01:50:50.552337+0000","last_became_peered":"2024-03-19T01:50:50.552337+0000","last_unstale":"2024-03-19T01:50:52.062889+0000","last_undegraded":"2024-03-19T01:50:52.062889+0000","last_fullsized":"2024-03-19T01:50:52.062889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:27.912513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062973+0000","last_change":"2024-03-19T01:50:49.554531+0000","last_active":"2024-03-19T01:50:52.062973+0000","last_peered":"2024-03-19T01:50:52.062973+0000","last_clean":"2024-03-19T01:50:52.062973+0000","last_became_active":"2024-03-19T01:50:49.554196+0000","last_became_peered":"2024-03-19T01:50:49.554196+0000","last_unstale":"2024-03-19T01:50:52.062973+0000","last_undegraded":"2024-03-19T01:50:52.062973+0000","last_fullsized":"2024-03-19T01:50:52.062973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:04.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062973+0000","last_change":"2024-03-19T01:50:50.558390+0000","last_active":"2024-03-19T01:50:52.062973+0000","last_peered":"2024-03-19T01:50:52.062973+0000","last_clean":"2024-03-19T01:50:52.062973+0000","last_became_active":"2024-03-19T01:50:50.558022+0000","last_became_peered":"2024-03-19T01:50:50.558022+0000","last_unstale":"2024-03-19T01:50:52.062973+0000","last_undegraded":"2024-03-19T01:50:52.062973+0000","last_fullsized":"2024-03-19T01:50:52.062973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:20.794803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549973+0000","last_change":"2024-03-19T01:50:49.553203+0000","last_active":"2024-03-19T01:50:51.549973+0000","last_peered":"2024-03-19T01:50:51.549973+0000","last_clean":"2024-03-19T01:50:51.549973+0000","last_became_active":"2024-03-19T01:50:49.552243+0000","last_became_peered":"2024-03-19T01:50:49.552243+0000","last_unstale":"2024-03-19T01:50:51.549973+0000","last_undegraded":"2024-03-19T01:50:51.549973+0000","last_fullsized":"2024-03-19T01:50:51.549973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:16.313075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063066+0000","last_change":"2024-03-19T01:50:50.567317+0000","last_active":"2024-03-19T01:50:52.063066+0000","last_peered":"2024-03-19T01:50:52.063066+0000","last_clean":"2024-03-19T01:50:52.063066+0000","last_became_active":"2024-03-19T01:50:50.566744+0000","last_became_peered":"2024-03-19T01:50:50.566744+0000","last_unstale":"2024-03-19T01:50:52.063066+0000","last_undegraded":"2024-03-19T01:50:52.063066+0000","last_fullsized":"2024-03-19T01:50:52.063066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:01:34.239148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062826+0000","last_change":"2024-03-19T01:50:50.571361+0000","last_active":"2024-03-19T01:50:52.062826+0000","last_peered":"2024-03-19T01:50:52.062826+0000","last_clean":"2024-03-19T01:50:52.062826+0000","last_became_active":"2024-03-19T01:50:50.570845+0000","last_became_peered":"2024-03-19T01:50:50.570845+0000","last_unstale":"2024-03-19T01:50:52.062826+0000","last_undegraded":"2024-03-19T01:50:52.062826+0000","last_fullsized":"2024-03-19T01:50:52.062826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:27:46.816815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549658+0000","last_change":"2024-03-19T01:50:50.564337+0000","last_active":"2024-03-19T01:50:51.549658+0000","last_peered":"2024-03-19T01:50:51.549658+0000","last_clean":"2024-03-19T01:50:51.549658+0000","last_became_active":"2024-03-19T01:50:50.564002+0000","last_became_peered":"2024-03-19T01:50:50.564002+0000","last_unstale":"2024-03-19T01:50:51.549658+0000","last_undegraded":"2024-03-19T01:50:51.549658+0000","last_fullsized":"2024-03-19T01:50:51.549658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:19:51.072829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.549914+0000","last_change":"2024-03-19T01:50:50.560962+0000","last_active":"2024-03-19T01:50:51.549914+0000","last_peered":"2024-03-19T01:50:51.549914+0000","last_clean":"2024-03-19T01:50:51.549914+0000","last_became_active":"2024-03-19T01:50:50.560427+0000","last_became_peered":"2024-03-19T01:50:50.560427+0000","last_unstale":"2024-03-19T01:50:51.549914+0000","last_undegraded":"2024-03-19T01:50:51.549914+0000","last_fullsized":"2024-03-19T01:50:51.549914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:54.472200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062303+0000","last_change":"2024-03-19T01:50:50.557806+0000","last_active":"2024-03-19T01:50:52.062303+0000","last_peered":"2024-03-19T01:50:52.062303+0000","last_clean":"2024-03-19T01:50:52.062303+0000","last_became_active":"2024-03-19T01:50:50.557494+0000","last_became_peered":"2024-03-19T01:50:50.557494+0000","last_unstale":"2024-03-19T01:50:52.062303+0000","last_undegraded":"2024-03-19T01:50:52.062303+0000","last_fullsized":"2024-03-19T01:50:52.062303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:35:53.787047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550340+0000","last_change":"2024-03-19T01:50:50.554414+0000","last_active":"2024-03-19T01:50:51.550340+0000","last_peered":"2024-03-19T01:50:51.550340+0000","last_clean":"2024-03-19T01:50:51.550340+0000","last_became_active":"2024-03-19T01:50:50.554010+0000","last_became_peered":"2024-03-19T01:50:50.554010+0000","last_unstale":"2024-03-19T01:50:51.550340+0000","last_undegraded":"2024-03-19T01:50:51.550340+0000","last_fullsized":"2024-03-19T01:50:51.550340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:02:54.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062855+0000","last_change":"2024-03-19T01:50:50.567759+0000","last_active":"2024-03-19T01:50:52.062855+0000","last_peered":"2024-03-19T01:50:52.062855+0000","last_clean":"2024-03-19T01:50:52.062855+0000","last_became_active":"2024-03-19T01:50:50.566529+0000","last_became_peered":"2024-03-19T01:50:50.566529+0000","last_unstale":"2024-03-19T01:50:52.062855+0000","last_undegraded":"2024-03-19T01:50:52.062855+0000","last_fullsized":"2024-03-19T01:50:52.062855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:25:05.002161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550007+0000","last_change":"2024-03-19T01:50:50.563915+0000","last_active":"2024-03-19T01:50:51.550007+0000","last_peered":"2024-03-19T01:50:51.550007+0000","last_clean":"2024-03-19T01:50:51.550007+0000","last_became_active":"2024-03-19T01:50:50.563596+0000","last_became_peered":"2024-03-19T01:50:50.563596+0000","last_unstale":"2024-03-19T01:50:51.550007+0000","last_undegraded":"2024-03-19T01:50:51.550007+0000","last_fullsized":"2024-03-19T01:50:51.550007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:12.832402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063117+0000","last_change":"2024-03-19T01:50:50.556507+0000","last_active":"2024-03-19T01:50:52.063117+0000","last_peered":"2024-03-19T01:50:52.063117+0000","last_clean":"2024-03-19T01:50:52.063117+0000","last_became_active":"2024-03-19T01:50:50.556197+0000","last_became_peered":"2024-03-19T01:50:50.556197+0000","last_unstale":"2024-03-19T01:50:52.063117+0000","last_undegraded":"2024-03-19T01:50:52.063117+0000","last_fullsized":"2024-03-19T01:50:52.063117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:25:26.919274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062737+0000","last_change":"2024-03-19T01:50:50.554403+0000","last_active":"2024-03-19T01:50:52.062737+0000","last_peered":"2024-03-19T01:50:52.062737+0000","last_clean":"2024-03-19T01:50:52.062737+0000","last_became_active":"2024-03-19T01:50:50.553985+0000","last_became_peered":"2024-03-19T01:50:50.553985+0000","last_unstale":"2024-03-19T01:50:52.062737+0000","last_undegraded":"2024-03-19T01:50:52.062737+0000","last_fullsized":"2024-03-19T01:50:52.062737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:50.923915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.062685+0000","last_change":"2024-03-19T01:50:50.554391+0000","last_active":"2024-03-19T01:50:52.062685+0000","last_peered":"2024-03-19T01:50:52.062685+0000","last_clean":"2024-03-19T01:50:52.062685+0000","last_became_active":"2024-03-19T01:50:50.552987+0000","last_became_peered":"2024-03-19T01:50:50.552987+0000","last_unstale":"2024-03-19T01:50:52.062685+0000","last_undegraded":"2024-03-19T01:50:52.062685+0000","last_fullsized":"2024-03-19T01:50:52.062685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:07.107613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.551956+0000","last_change":"2024-03-19T01:50:50.556172+0000","last_active":"2024-03-19T01:50:51.551956+0000","last_peered":"2024-03-19T01:50:51.551956+0000","last_clean":"2024-03-19T01:50:51.551956+0000","last_became_active":"2024-03-19T01:50:50.555468+0000","last_became_peered":"2024-03-19T01:50:50.555468+0000","last_unstale":"2024-03-19T01:50:51.551956+0000","last_undegraded":"2024-03-19T01:50:51.551956+0000","last_fullsized":"2024-03-19T01:50:51.551956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:28:03.005962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:51.550166+0000","last_change":"2024-03-19T01:50:50.550915+0000","last_active":"2024-03-19T01:50:51.550166+0000","last_peered":"2024-03-19T01:50:51.550166+0000","last_clean":"2024-03-19T01:50:51.550166+0000","last_became_active":"2024-03-19T01:50:50.550535+0000","last_became_peered":"2024-03-19T01:50:50.550535+0000","last_unstale":"2024-03-19T01:50:51.550166+0000","last_undegraded":"2024-03-19T01:50:51.550166+0000","last_fullsized":"2024-03-19T01:50:51.550166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:05:21.959483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-19T01:50:52.063005+0000","last_change":"2024-03-19T01:50:50.567475+0000","last_active":"2024-03-19T01:50:52.063005+0000","last_peered":"2024-03-19T01:50:52.063005+0000","last_clean":"2024-03-19T01:50:52.063005+0000","last_became_active":"2024-03-19T01:50:50.567103+0000","last_became_peered":"2024-03-19T01:50:50.567103+0000","last_unstale":"2024-03-19T01:50:52.063005+0000","last_undegraded":"2024-03-19T01:50:52.063005+0000","last_fullsized":"2024-03-19T01:50:52.063005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:41.866610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244932,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5222,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":143,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":54049,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":1031457,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":439643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":1030681,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":461779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":438951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1029071,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":40556,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:50:58.804 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-19T01:50:58.804 INFO:tasks.cephadm:Setting up client nodes... 2024-03-19T01:50:58.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-19T01:50:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:59 smithi092 ceph-mon[62574]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-19T01:50:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:59 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/508377502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:50:59 smithi092 ceph-mon[62574]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:50:59.130 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:50:59.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:59 smithi144 ceph-mon[67199]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-19T01:50:59.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:59 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/508377502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:59.347 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:50:59 smithi144 ceph-mon[67199]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:50:59.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:59 smithi159 ceph-mon[67110]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-19T01:50:59.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:59 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/508377502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:50:59.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:50:59 smithi159 ceph-mon[67110]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:51:00.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:00 smithi092 ceph-mon[62574]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:51:00.310 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:00 smithi092 ceph-mon[62574]: pgmap v295: 97 pgs: 97 active+clean; 619 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-03-19T01:51:00.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:00 smithi144 ceph-mon[67199]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:51:00.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:00 smithi144 ceph-mon[67199]: pgmap v295: 97 pgs: 97 active+clean; 619 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-03-19T01:51:00.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:00 smithi159 ceph-mon[67110]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:51:00.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:00 smithi159 ceph-mon[67110]: pgmap v295: 97 pgs: 97 active+clean; 619 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-03-19T01:51:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 -- 172.21.15.92:0/3893266624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 msgr2=0x7f38a410a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3893266624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 0x7f38a410a9a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f389c00b0a0 tx=0x7f389c00b3b0 comp rx=0 tx=0).stop 2024-03-19T01:51:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 -- 172.21.15.92:0/3893266624 shutdown_connections 2024-03-19T01:51:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3893266624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 0x7f38a410a9a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3893266624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a41039b0 0x7f38a4103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3893266624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a4100430 0x7f38a4100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 -- 172.21.15.92:0/3893266624 >> 172.21.15.92:0/3893266624 conn(0x7f38a40fbfb0 msgr2=0x7f38a40fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 -- 172.21.15.92:0/3893266624 shutdown_connections 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 -- 172.21.15.92:0/3893266624 wait complete. 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 Processor -- start 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.898+0000 7f38a99e7700 1 -- start start 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a99e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a4100430 0x7f38a4198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a99e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 0x7f38a4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a99e7700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a41039b0 0x7f38a419fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a99e7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f38a410dfb0 con 0x7f38a4100dd0 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a99e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38a410de30 con 0x7f38a4100430 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 0x7f38a4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a99e7700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38a410e130 con 0x7f38a41039b0 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 0x7f38a4198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37966/0 (socket says 172.21.15.92:37966) 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a27fc700 1 -- 172.21.15.92:0/3713008092 learned_addr learned my addr 172.21.15.92:0/3713008092 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a4100430 0x7f38a4198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a37fe700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a41039b0 0x7f38a419fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a27fc700 1 -- 172.21.15.92:0/3713008092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a41039b0 msgr2=0x7f38a419fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a27fc700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a41039b0 0x7f38a419fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a27fc700 1 -- 172.21.15.92:0/3713008092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a4100430 msgr2=0x7f38a4198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a27fc700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a4100430 0x7f38a4198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a27fc700 1 -- 172.21.15.92:0/3713008092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a41a00b0 con 0x7f38a4100dd0 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a37fe700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a41039b0 0x7f38a419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a2ffd700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a4100430 0x7f38a4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.899+0000 7f38a27fc700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 0x7f38a4198990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f389800e9f0 tx=0x7f389800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.900+0000 7f38a89e5700 1 -- 172.21.15.92:0/3713008092 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389800cbd0 con 0x7f38a4100dd0 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.900+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a41a03a0 con 0x7f38a4100dd0 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.900+0000 7f38a89e5700 1 -- 172.21.15.92:0/3713008092 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389800ee90 con 0x7f38a4100dd0 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.900+0000 7f38a89e5700 1 -- 172.21.15.92:0/3713008092 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980056c0 con 0x7f38a4100dd0 2024-03-19T01:51:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.900+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a41a0890 con 0x7f38a4100dd0 2024-03-19T01:51:00.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.901+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a4050820 con 0x7f38a4100dd0 2024-03-19T01:51:00.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.902+0000 7f38a89e5700 1 -- 172.21.15.92:0/3713008092 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f3898004750 con 0x7f38a4100dd0 2024-03-19T01:51:00.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.902+0000 7f38a89e5700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f388c0a29d0 0x7f388c0a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:00.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.903+0000 7f38a2ffd700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f388c0a29d0 0x7f388c0a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:00.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.903+0000 7f38a89e5700 1 -- 172.21.15.92:0/3713008092 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f38980cec70 con 0x7f38a4100dd0 2024-03-19T01:51:00.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.903+0000 7f38a2ffd700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f388c0a29d0 0x7f388c0a4e90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38a406a6f0 tx=0x7f389400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:00.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:00.908+0000 7f38a89e5700 1 -- 172.21.15.92:0/3713008092 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f389809bd60 con 0x7f38a4100dd0 2024-03-19T01:51:01.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.199+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f38a402dbb0 con 0x7f38a4100dd0 2024-03-19T01:51:01.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.201+0000 7f38a89e5700 1 -- 172.21.15.92:0/3713008092 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f389809b660 con 0x7f38a4100dd0 2024-03-19T01:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-03-19T01:51:01.202 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCF7/hlll8ADBAAix6Bqy7f0AZKuafrm6iLYw== 2024-03-19T01:51:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.204+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f388c0a29d0 msgr2=0x7f388c0a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.204+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f388c0a29d0 0x7f388c0a4e90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38a406a6f0 tx=0x7f389400b410 comp rx=0 tx=0).stop 2024-03-19T01:51:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.204+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 msgr2=0x7f38a4198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.204+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 0x7f38a4198990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f389800e9f0 tx=0x7f389800ed00 comp rx=0 tx=0).stop 2024-03-19T01:51:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.205+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 shutdown_connections 2024-03-19T01:51:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.205+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f388c0a29d0 0x7f388c0a4e90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.205+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a41039b0 0x7f38a419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:01.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.205+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38a4100430 0x7f38a4198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:01.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.205+0000 7f38a99e7700 1 --2- 172.21.15.92:0/3713008092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38a4100dd0 0x7f38a4198990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:01.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.205+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 >> 172.21.15.92:0/3713008092 conn(0x7f38a40fbfb0 msgr2=0x7f38a4106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:01.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.205+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 shutdown_connections 2024-03-19T01:51:01.206 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:01.205+0000 7f38a99e7700 1 -- 172.21.15.92:0/3713008092 wait complete. 2024-03-19T01:51:01.206 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:51:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: 2024-03-19T01:51:00.862+0000 7f2ab486a700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-19T01:51:01.206 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 01:51:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: 2024-03-19T01:51:00.862+0000 7f2ab486a700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-19T01:51:01.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:01 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3713008092' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T01:51:01.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:01 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3713008092' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-19T01:51:01.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:01 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3713008092' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T01:51:01.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:01 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3713008092' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-19T01:51:01.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:01 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3713008092' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-19T01:51:01.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:01 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3713008092' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-19T01:51:01.932 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:51:01.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-19T01:51:01.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-19T01:51:01.974 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-19T01:51:01.974 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-19T01:51:01.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-19T01:51:02.267 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:02.520 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:02 smithi092 ceph-mon[62574]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-03-19T01:51:02.564 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:02 smithi159 ceph-mon[67110]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-03-19T01:51:02.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:02 smithi144 ceph-mon[67199]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-03-19T01:51:03.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.980+0000 7f4fc1e58700 1 -- 172.21.15.92:0/1382432857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc107630 msgr2=0x7f4fbc10f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.980+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/1382432857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc107630 0x7f4fbc10f200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400b0d0 tx=0x7f4fb400b3e0 comp rx=0 tx=0).stop 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.981+0000 7f4fc1e58700 1 -- 172.21.15.92:0/1382432857 shutdown_connections 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.981+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/1382432857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc107630 0x7f4fbc10f200 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.981+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/1382432857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fbc102550 0x7f4fbc102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.981+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/1382432857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fbc102f00 0x7f4fbc106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.981+0000 7f4fc1e58700 1 -- 172.21.15.92:0/1382432857 >> 172.21.15.92:0/1382432857 conn(0x7f4fbc0fddc0 msgr2=0x7f4fbc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.981+0000 7f4fc1e58700 1 -- 172.21.15.92:0/1382432857 shutdown_connections 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.981+0000 7f4fc1e58700 1 -- 172.21.15.92:0/1382432857 wait complete. 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.981+0000 7f4fc1e58700 1 Processor -- start 2024-03-19T01:51:03.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fc1e58700 1 -- start start 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fc1e58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc102550 0x7f4fbc19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fc1e58700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fbc102f00 0x7f4fbc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fc1e58700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fbc107630 0x7f4fbc198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fc1e58700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc112640 con 0x7f4fbc102550 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fc1e58700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1124c0 con 0x7f4fbc107630 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fc1e58700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1127c0 con 0x7f4fbc102f00 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc102550 0x7f4fbc19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc102550 0x7f4fbc19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48930/0 (socket says 172.21.15.92:48930) 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbb7fe700 1 -- 172.21.15.92:0/3850123288 learned_addr learned my addr 172.21.15.92:0/3850123288 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbaffd700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fbc102f00 0x7f4fbc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:03.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbbfff700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fbc107630 0x7f4fbc198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbb7fe700 1 -- 172.21.15.92:0/3850123288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fbc102f00 msgr2=0x7f4fbc19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbb7fe700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fbc102f00 0x7f4fbc19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbb7fe700 1 -- 172.21.15.92:0/3850123288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fbc107630 msgr2=0x7f4fbc198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbb7fe700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fbc107630 0x7f4fbc198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbb7fe700 1 -- 172.21.15.92:0/3850123288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc198fc0 con 0x7f4fbc102550 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbaffd700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fbc102f00 0x7f4fbc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.982+0000 7f4fbb7fe700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc102550 0x7f4fbc19e6a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00e9c0 tx=0x7f4fac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.983+0000 7f4fb8ff9700 1 -- 172.21.15.92:0/3850123288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00cbd0 con 0x7f4fbc102550 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.983+0000 7f4fb8ff9700 1 -- 172.21.15.92:0/3850123288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fac00ee60 con 0x7f4fbc102550 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.983+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc1992b0 con 0x7f4fbc102550 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.983+0000 7f4fb8ff9700 1 -- 172.21.15.92:0/3850123288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac005630 con 0x7f4fbc102550 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.983+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc1a5570 con 0x7f4fbc102550 2024-03-19T01:51:03.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.984+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fbc068fd0 con 0x7f4fbc102550 2024-03-19T01:51:03.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.985+0000 7f4fb8ff9700 1 -- 172.21.15.92:0/3850123288 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4fac004750 con 0x7f4fbc102550 2024-03-19T01:51:03.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.985+0000 7f4fb8ff9700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4fa40a29a0 0x7f4fa40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:03.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.986+0000 7f4fbaffd700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4fa40a29a0 0x7f4fa40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:03.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.986+0000 7f4fb8ff9700 1 -- 172.21.15.92:0/3850123288 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4fac0cefd0 con 0x7f4fbc102550 2024-03-19T01:51:03.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.986+0000 7f4fbaffd700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4fa40a29a0 0x7f4fa40a4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc108bb0 tx=0x7f4fb00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:03.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:03.989+0000 7f4fb8ff9700 1 -- 172.21.15.92:0/3850123288 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4fac053740 con 0x7f4fbc102550 2024-03-19T01:51:04.247 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.246+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f4fbc02dbb0 con 0x7f4fbc102550 2024-03-19T01:51:04.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.253+0000 7f4fb8ff9700 1 -- 172.21.15.92:0/3850123288 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+262660 (secure 0 0 0) 0x7f4fac09bfc0 con 0x7f4fbc102550 2024-03-19T01:51:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4fa40a29a0 msgr2=0x7f4fa40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4fa40a29a0 0x7f4fa40a4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc108bb0 tx=0x7f4fb00073d0 comp rx=0 tx=0).stop 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc102550 msgr2=0x7f4fbc19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc102550 0x7f4fbc19e6a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00e9c0 tx=0x7f4fac00ecd0 comp rx=0 tx=0).stop 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 shutdown_connections 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4fa40a29a0 0x7f4fa40a4e60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fbc107630 0x7f4fbc198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fbc102550 0x7f4fbc19e6a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 --2- 172.21.15.92:0/3850123288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fbc102f00 0x7f4fbc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 >> 172.21.15.92:0/3850123288 conn(0x7f4fbc0fddc0 msgr2=0x7f4fbc0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 shutdown_connections 2024-03-19T01:51:04.259 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:04.257+0000 7f4fc1e58700 1 -- 172.21.15.92:0/3850123288 wait complete. 2024-03-19T01:51:04.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:04 smithi092 ceph-mon[62574]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-03-19T01:51:04.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:04 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3850123288' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-19T01:51:04.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:04 smithi144 ceph-mon[67199]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-03-19T01:51:04.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:04 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3850123288' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-19T01:51:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:04 smithi159 ceph-mon[67110]: pgmap v297: 97 pgs: 97 active+clean; 621 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-03-19T01:51:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:04 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3850123288' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-19T01:51:04.975 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3230736370},{"type":"v1","addr":"172.21.15.92:6801","nonce":3230736370}]},"active_addr":"172.21.15.92:6801/3230736370","active_change":"2024-03-19T01:43:39.363264+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.92:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2292169595}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2477781456}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":206244231}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":4250800294}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":4099768894}]}]} 2024-03-19T01:51:04.980 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-19T01:51:04.980 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-19T01:51:04.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-19T01:51:05.230 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:06.807 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:06 smithi092 ceph-mon[62574]: pgmap v298: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 KiB/s wr, 23 op/s 2024-03-19T01:51:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:06 smithi144 ceph-mon[67199]: pgmap v298: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 KiB/s wr, 23 op/s 2024-03-19T01:51:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:06 smithi159 ceph-mon[67110]: pgmap v298: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.7 KiB/s wr, 23 op/s 2024-03-19T01:51:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.038+0000 7f5ef27c0700 1 -- 172.21.15.92:0/3700319307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06ac10 msgr2=0x7f5eec06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.038+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/3700319307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06ac10 0x7f5eec06aff0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0099c0 tx=0x7f5edc009cd0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.038+0000 7f5ef27c0700 1 -- 172.21.15.92:0/3700319307 shutdown_connections 2024-03-19T01:51:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.038+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/3700319307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eec107680 0x7f5eec10f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.038+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/3700319307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06ac10 0x7f5eec06aff0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.038+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/3700319307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec06b5c0 0x7f5eec106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.038+0000 7f5ef27c0700 1 -- 172.21.15.92:0/3700319307 >> 172.21.15.92:0/3700319307 conn(0x7f5eec078180 msgr2=0x7f5eec078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.039+0000 7f5ef27c0700 1 -- 172.21.15.92:0/3700319307 shutdown_connections 2024-03-19T01:51:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.039+0000 7f5ef27c0700 1 -- 172.21.15.92:0/3700319307 wait complete. 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.039+0000 7f5ef27c0700 1 Processor -- start 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.039+0000 7f5ef27c0700 1 -- start start 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.039+0000 7f5ef27c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec06ac10 0x7f5eec19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ef27c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06b5c0 0x7f5eec19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ef27c0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eec107680 0x7f5eec1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ef27c0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec112610 con 0x7f5eec06b5c0 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ef27c0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec112490 con 0x7f5eec06ac10 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ef27c0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec112790 con 0x7f5eec107680 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eeb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06b5c0 0x7f5eec19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eeb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06b5c0 0x7f5eec19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48948/0 (socket says 172.21.15.92:48948) 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eeb7fe700 1 -- 172.21.15.92:0/1011818255 learned_addr learned my addr 172.21.15.92:0/1011818255 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ef0d5d700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eec107680 0x7f5eec1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eebfff700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec06ac10 0x7f5eec19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:07.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eeb7fe700 1 -- 172.21.15.92:0/1011818255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eec107680 msgr2=0x7f5eec1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eeb7fe700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eec107680 0x7f5eec1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eeb7fe700 1 -- 172.21.15.92:0/1011818255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec06ac10 msgr2=0x7f5eec19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eeb7fe700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec06ac10 0x7f5eec19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eeb7fe700 1 -- 172.21.15.92:0/1011818255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec199010 con 0x7f5eec06b5c0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5eeb7fe700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06b5c0 0x7f5eec19ec30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000d950 tx=0x7f5ee000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ee97fa700 1 -- 172.21.15.92:0/1011818255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0016070 con 0x7f5eec06b5c0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eec199300 con 0x7f5eec06b5c0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ee97fa700 1 -- 172.21.15.92:0/1011818255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee0004d10 con 0x7f5eec06b5c0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ee97fa700 1 -- 172.21.15.92:0/1011818255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00056a0 con 0x7f5eec06b5c0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.040+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eec1a55c0 con 0x7f5eec06b5c0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.041+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eec10a850 con 0x7f5eec06b5c0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.043+0000 7f5ee97fa700 1 -- 172.21.15.92:0/1011818255 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f5ee0004220 con 0x7f5eec06b5c0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.043+0000 7f5ee97fa700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ed40a29a0 0x7f5ed40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.043+0000 7f5eebfff700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ed40a29a0 0x7f5ed40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:07.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.044+0000 7f5ee97fa700 1 -- 172.21.15.92:0/1011818255 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f5ee00cec40 con 0x7f5eec06b5c0 2024-03-19T01:51:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.044+0000 7f5eebfff700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ed40a29a0 0x7f5ed40a4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5edc000c00 tx=0x7f5edc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:07.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.047+0000 7f5ee97fa700 1 -- 172.21.15.92:0/1011818255 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5ee001f020 con 0x7f5eec06b5c0 2024-03-19T01:51:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.249+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5eec02dbb0 con 0x7f5eec06b5c0 2024-03-19T01:51:07.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.250+0000 7f5ee97fa700 1 -- 172.21.15.92:0/1011818255 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22256 (secure 0 0 0) 0x7f5ee009bd30 con 0x7f5eec06b5c0 2024-03-19T01:51:07.251 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:51:07.251 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":85,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:50:51.542012+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:51:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.252+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ed40a29a0 msgr2=0x7f5ed40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.252+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ed40a29a0 0x7f5ed40a4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5edc000c00 tx=0x7f5edc005fb0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.252+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06b5c0 msgr2=0x7f5eec19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.252+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06b5c0 0x7f5eec19ec30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000d950 tx=0x7f5ee000dc60 comp rx=0 tx=0).stop 2024-03-19T01:51:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.253+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 shutdown_connections 2024-03-19T01:51:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.253+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f5ed40a29a0 0x7f5ed40a4e60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.253+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eec107680 0x7f5eec1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.253+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eec06ac10 0x7f5eec19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.253+0000 7f5ef27c0700 1 --2- 172.21.15.92:0/1011818255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eec06b5c0 0x7f5eec19ec30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.253+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 >> 172.21.15.92:0/1011818255 conn(0x7f5eec078180 msgr2=0x7f5eec0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.253+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 shutdown_connections 2024-03-19T01:51:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:07.253+0000 7f5ef27c0700 1 -- 172.21.15.92:0/1011818255 wait complete. 2024-03-19T01:51:07.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:07 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1011818255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:51:07.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:07 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1011818255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:51:07.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:07 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1011818255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:51:08.045 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-19T01:51:08.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-19T01:51:08.298 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:08.723 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:08 smithi092 ceph-mon[62574]: pgmap v299: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.3 KiB/s wr, 23 op/s 2024-03-19T01:51:08.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:08 smithi144 ceph-mon[67199]: pgmap v299: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.3 KiB/s wr, 23 op/s 2024-03-19T01:51:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:08 smithi159 ceph-mon[67110]: pgmap v299: 97 pgs: 97 active+clean; 626 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.3 KiB/s wr, 23 op/s 2024-03-19T01:51:10.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.125+0000 7fb3681ec700 1 -- 172.21.15.92:0/2804527329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101210 msgr2=0x7fb360101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:10.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.125+0000 7fb3681ec700 1 --2- 172.21.15.92:0/2804527329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101210 0x7fb360101670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb350009990 tx=0x7fb350009ca0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.125+0000 7fb3681ec700 1 -- 172.21.15.92:0/2804527329 shutdown_connections 2024-03-19T01:51:10.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.125+0000 7fb3681ec700 1 --2- 172.21.15.92:0/2804527329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb360101bb0 0x7fb36010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.125+0000 7fb3681ec700 1 --2- 172.21.15.92:0/2804527329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb360107250 0x7fb360107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.125+0000 7fb3681ec700 1 --2- 172.21.15.92:0/2804527329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101210 0x7fb360101670 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.125+0000 7fb3681ec700 1 -- 172.21.15.92:0/2804527329 >> 172.21.15.92:0/2804527329 conn(0x7fb360078140 msgr2=0x7fb360078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.125+0000 7fb3681ec700 1 -- 172.21.15.92:0/2804527329 shutdown_connections 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.125+0000 7fb3681ec700 1 -- 172.21.15.92:0/2804527329 wait complete. 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb3681ec700 1 Processor -- start 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb3681ec700 1 -- start start 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb3681ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb360101210 0x7fb36019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb3681ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101bb0 0x7fb36019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb3681ec700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb360107250 0x7fb360197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb3681ec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb3601114b0 con 0x7fb360101bb0 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb3681ec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb360111330 con 0x7fb360101210 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb3681ec700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb360111630 con 0x7fb360107250 2024-03-19T01:51:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb365787700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101bb0 0x7fb36019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb365787700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101bb0 0x7fb36019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48968/0 (socket says 172.21.15.92:48968) 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb365787700 1 -- 172.21.15.92:0/1122297098 learned_addr learned my addr 172.21.15.92:0/1122297098 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb365f88700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb360101210 0x7fb36019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb366789700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb360107250 0x7fb360197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb365787700 1 -- 172.21.15.92:0/1122297098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb360107250 msgr2=0x7fb360197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.126+0000 7fb365787700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb360107250 0x7fb360197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb365787700 1 -- 172.21.15.92:0/1122297098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb360101210 msgr2=0x7fb36019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb365787700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb360101210 0x7fb36019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb365787700 1 -- 172.21.15.92:0/1122297098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb360197f70 con 0x7fb360101bb0 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb365f88700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb360101210 0x7fb36019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb366789700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb360107250 0x7fb360197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb365787700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101bb0 0x7fb36019db50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb350000c00 tx=0x7fb35000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb356ffd700 1 -- 172.21.15.92:0/1122297098 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350029070 con 0x7fb360101bb0 2024-03-19T01:51:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb360198200 con 0x7fb360101bb0 2024-03-19T01:51:10.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb356ffd700 1 -- 172.21.15.92:0/1122297098 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb350003d70 con 0x7fb360101bb0 2024-03-19T01:51:10.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb356ffd700 1 -- 172.21.15.92:0/1122297098 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3500246d0 con 0x7fb360101bb0 2024-03-19T01:51:10.129 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.127+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb360069fc0 con 0x7fb360101bb0 2024-03-19T01:51:10.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.128+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb360102900 con 0x7fb360101bb0 2024-03-19T01:51:10.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.129+0000 7fb356ffd700 1 -- 172.21.15.92:0/1122297098 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb35002c070 con 0x7fb360101bb0 2024-03-19T01:51:10.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.130+0000 7fb356ffd700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb34c0a2920 0x7fb34c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:10.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.130+0000 7fb365f88700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb34c0a2920 0x7fb34c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:10.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.130+0000 7fb356ffd700 1 -- 172.21.15.92:0/1122297098 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb3500d1d70 con 0x7fb360101bb0 2024-03-19T01:51:10.130 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.130+0000 7fb365f88700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb34c0a2920 0x7fb34c0a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009fd0 tx=0x7fb35c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:10.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.134+0000 7fb356ffd700 1 -- 172.21.15.92:0/1122297098 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb350013070 con 0x7fb360101bb0 2024-03-19T01:51:10.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.343+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb36006a9f0 con 0x7fb360101bb0 2024-03-19T01:51:10.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.345+0000 7fb356ffd700 1 -- 172.21.15.92:0/1122297098 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22256 (secure 0 0 0) 0x7fb35009eef0 con 0x7fb360101bb0 2024-03-19T01:51:10.344 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:51:10.345 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":85,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:50:51.542012+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb34c0a2920 msgr2=0x7fb34c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb34c0a2920 0x7fb34c0a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009fd0 tx=0x7fb35c009260 comp rx=0 tx=0).stop 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101bb0 msgr2=0x7fb36019db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101bb0 0x7fb36019db50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb350000c00 tx=0x7fb35000bab0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 shutdown_connections 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb34c0a2920 0x7fb34c0a4de0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb360107250 0x7fb360197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb360101210 0x7fb36019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 --2- 172.21.15.92:0/1122297098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb360101bb0 0x7fb36019db50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 >> 172.21.15.92:0/1122297098 conn(0x7fb360078140 msgr2=0x7fb3600ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 shutdown_connections 2024-03-19T01:51:10.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:10.347+0000 7fb3681ec700 1 -- 172.21.15.92:0/1122297098 wait complete. 2024-03-19T01:51:10.587 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:10 smithi092 ceph-mon[62574]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 20 op/s 2024-03-19T01:51:10.587 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:10 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1122297098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:51:10.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:10 smithi144 ceph-mon[67199]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 20 op/s 2024-03-19T01:51:10.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:10 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1122297098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:51:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:10 smithi159 ceph-mon[67110]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 20 op/s 2024-03-19T01:51:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:10 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1122297098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:51:11.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-19T01:51:11.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-19T01:51:11.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-19T01:51:11.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-19T01:51:11.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-19T01:51:11.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-19T01:51:11.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-19T01:51:11.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-19T01:51:11.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-19T01:51:11.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-19T01:51:11.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-19T01:51:11.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-19T01:51:11.884 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:11.896 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:11.908 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:11.911 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:11.915 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:11.918 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:11.922 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:12.798 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:12 smithi092 ceph-mon[62574]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-03-19T01:51:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:12 smithi144 ceph-mon[67199]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-03-19T01:51:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:12 smithi159 ceph-mon[67110]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-03-19T01:51:14.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:14 smithi092 ceph-mon[62574]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-19T01:51:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:14 smithi144 ceph-mon[67199]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-19T01:51:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:14 smithi159 ceph-mon[67110]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-03-19T01:51:16.801 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:16 smithi092 ceph-mon[62574]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-03-19T01:51:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:16 smithi144 ceph-mon[67199]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-03-19T01:51:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:16 smithi159 ceph-mon[67110]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-03-19T01:51:17.748 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:17.749 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:17.750 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:18.762 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:18.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:18 smithi092 ceph-mon[62574]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-19T01:51:18.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:18 smithi144 ceph-mon[67199]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-19T01:51:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:18 smithi159 ceph-mon[67110]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.097+0000 7fb11df2d700 1 -- 172.21.15.92:0/3852280546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb118108470 msgr2=0x7fb118108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.097+0000 7fb11df2d700 1 --2- 172.21.15.92:0/3852280546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb118108470 0x7fb118108850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb108009990 tx=0x7fb108009ca0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.097+0000 7fb11df2d700 1 -- 172.21.15.92:0/3852280546 shutdown_connections 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.097+0000 7fb11df2d700 1 --2- 172.21.15.92:0/3852280546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb118102d50 0x7fb11810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.097+0000 7fb11df2d700 1 --2- 172.21.15.92:0/3852280546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb118108470 0x7fb118108850 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.097+0000 7fb11df2d700 1 --2- 172.21.15.92:0/3852280546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb118102390 0x7fb118102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.097+0000 7fb11df2d700 1 -- 172.21.15.92:0/3852280546 >> 172.21.15.92:0/3852280546 conn(0x7fb1180793d0 msgr2=0x7fb1180797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.097+0000 7fb11df2d700 1 -- 172.21.15.92:0/3852280546 shutdown_connections 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.097+0000 7fb11df2d700 1 -- 172.21.15.92:0/3852280546 wait complete. 2024-03-19T01:51:20.100 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb11df2d700 1 Processor -- start 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb11df2d700 1 -- start start 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb11df2d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb118102390 0x7fb11819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb11df2d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb118102d50 0x7fb11819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb11df2d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb118108470 0x7fb118198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb11df2d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181125f0 con 0x7fb118108470 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb11df2d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb118112470 con 0x7fb118102d50 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb11df2d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb118112770 con 0x7fb118102390 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb1177fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb118102390 0x7fb11819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb117fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb118108470 0x7fb118198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb1177fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb118102390 0x7fb11819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:47580/0 (socket says 172.21.15.92:47580) 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb1177fe700 1 -- 172.21.15.92:0/2316543562 learned_addr learned my addr 172.21.15.92:0/2316543562 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.098+0000 7fb116ffd700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb118102d50 0x7fb11819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb1177fe700 1 -- 172.21.15.92:0/2316543562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb118102d50 msgr2=0x7fb11819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb1177fe700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb118102d50 0x7fb11819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb1177fe700 1 -- 172.21.15.92:0/2316543562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb118108470 msgr2=0x7fb118198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.101 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb1177fe700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb118108470 0x7fb118198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb1177fe700 1 -- 172.21.15.92:0/2316543562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb118198ff0 con 0x7fb118102390 2024-03-19T01:51:20.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb117fff700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb118108470 0x7fb118198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:20.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb1177fe700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb118102390 0x7fb11819e6d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb108000c00 tx=0x7fb10800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:20.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb114ff9700 1 -- 172.21.15.92:0/2316543562 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108027070 con 0x7fb118102390 2024-03-19T01:51:20.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb114ff9700 1 -- 172.21.15.92:0/2316543562 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb108003d70 con 0x7fb118102390 2024-03-19T01:51:20.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb118199280 con 0x7fb118102390 2024-03-19T01:51:20.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb114ff9700 1 -- 172.21.15.92:0/2316543562 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10800f460 con 0x7fb118102390 2024-03-19T01:51:20.102 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.099+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb118074540 con 0x7fb118102390 2024-03-19T01:51:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.100+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb118062aa0 con 0x7fb118102390 2024-03-19T01:51:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.102+0000 7fb114ff9700 1 -- 172.21.15.92:0/2316543562 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb10802a070 con 0x7fb118102390 2024-03-19T01:51:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.103+0000 7fb114ff9700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1000a2920 0x7fb1000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.103+0000 7fb116ffd700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1000a2920 0x7fb1000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.103 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.103+0000 7fb114ff9700 1 -- 172.21.15.92:0/2316543562 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb1080cfdd0 con 0x7fb118102390 2024-03-19T01:51:20.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.103+0000 7fb114ff9700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] conn(0x7fb1000ac100 0x7fb1000ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.103+0000 7fb114ff9700 1 -- 172.21.15.92:0/2316543562 --> [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1080056d0 con 0x7fb1000ac100 2024-03-19T01:51:20.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.103+0000 7fb114ff9700 1 -- 172.21.15.92:0/2316543562 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb1080d02b0 con 0x7fb118102390 2024-03-19T01:51:20.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.103+0000 7fb116ffd700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1000a2920 0x7fb1000a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb118079a40 tx=0x7fb10c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:20.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.103+0000 7fb117fff700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] conn(0x7fb1000ac100 0x7fb1000ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.104+0000 7fb117fff700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] conn(0x7fb1000ac100 0x7fb1000ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:20.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.104+0000 7fb114ff9700 1 -- 172.21.15.92:0/2316543562 <== osd.4 v2:172.21.15.144:6808/3335611913 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1080056d0 con 0x7fb1000ac100 2024-03-19T01:51:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 --> [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb118062f20 con 0x7fb1000ac100 2024-03-19T01:51:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb114ff9700 1 -- 172.21.15.92:0/2316543562 <== osd.4 v2:172.21.15.144:6808/3335611913 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb118062f20 con 0x7fb1000ac100 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 >> [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] conn(0x7fb1000ac100 msgr2=0x7fb1000ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb11df2d700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] conn(0x7fb1000ac100 0x7fb1000ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1000a2920 msgr2=0x7fb1000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb11df2d700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb1000a2920 0x7fb1000a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb118079a40 tx=0x7fb10c009260 comp rx=0 tx=0).stop 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb118102390 msgr2=0x7fb11819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb11df2d700 1 --2- 172.21.15.92:0/2316543562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb118102390 0x7fb11819e6d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb108000c00 tx=0x7fb10800bab0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb117fff700 1 -- 172.21.15.92:0/2316543562 reap_dead start 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 shutdown_connections 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.121+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 >> 172.21.15.92:0/2316543562 conn(0x7fb1180793d0 msgr2=0x7fb1181010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.122+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 shutdown_connections 2024-03-19T01:51:20.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.122+0000 7fb11df2d700 1 -- 172.21.15.92:0/2316543562 wait complete. 2024-03-19T01:51:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.304+0000 7f39302dd700 1 -- 172.21.15.92:0/540789235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3928107690 msgr2=0x7f392810f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.304+0000 7f39302dd700 1 --2- 172.21.15.92:0/540789235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3928107690 0x7f392810f210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f392000b0d0 tx=0x7f392000b3e0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.304+0000 7f39302dd700 1 -- 172.21.15.92:0/540789235 shutdown_connections 2024-03-19T01:51:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.304+0000 7f39302dd700 1 --2- 172.21.15.92:0/540789235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3928107690 0x7f392810f210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.304+0000 7f39302dd700 1 --2- 172.21.15.92:0/540789235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f392806aab0 0x7f392806ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.304+0000 7f39302dd700 1 --2- 172.21.15.92:0/540789235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f392806b460 0x7f3928106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.304+0000 7f39302dd700 1 -- 172.21.15.92:0/540789235 >> 172.21.15.92:0/540789235 conn(0x7f3928077260 msgr2=0x7f3928077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.304+0000 7f39302dd700 1 -- 172.21.15.92:0/540789235 shutdown_connections 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.304+0000 7f39302dd700 1 -- 172.21.15.92:0/540789235 wait complete. 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f39302dd700 1 Processor -- start 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f39302dd700 1 -- start start 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f39302dd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f392806aab0 0x7f392819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f39302dd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f392806b460 0x7f392819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f39302dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3928107690 0x7f3928198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f39302dd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3928112650 con 0x7f3928107690 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f39302dd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39281124d0 con 0x7f392806aab0 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f39302dd700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39281127d0 con 0x7f392806b460 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f392e87a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3928107690 0x7f3928198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f392e87a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3928107690 0x7f3928198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58306/0 (socket says 172.21.15.92:58306) 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f392e87a700 1 -- 172.21.15.92:0/1327464033 learned_addr learned my addr 172.21.15.92:0/1327464033 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f392e079700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f392806aab0 0x7f392819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.305+0000 7f392d878700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f392806b460 0x7f392819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f392e87a700 1 -- 172.21.15.92:0/1327464033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f392806b460 msgr2=0x7f392819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f392e87a700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f392806b460 0x7f392819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f392e87a700 1 -- 172.21.15.92:0/1327464033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f392806aab0 msgr2=0x7f392819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f392e87a700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f392806aab0 0x7f392819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f392e87a700 1 -- 172.21.15.92:0/1327464033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3928199070 con 0x7f3928107690 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f392e079700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f392806aab0 0x7f392819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f392e87a700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3928107690 0x7f3928198910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3920006070 tx=0x7f3920007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f391f7fe700 1 -- 172.21.15.92:0/1327464033 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920003c00 con 0x7f3928107690 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3928199300 con 0x7f3928107690 2024-03-19T01:51:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f391f7fe700 1 -- 172.21.15.92:0/1327464033 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3920022070 con 0x7f3928107690 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f391f7fe700 1 -- 172.21.15.92:0/1327464033 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392001b6c0 con 0x7f3928107690 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.306+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39281a5620 con 0x7f3928107690 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.307+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3928062a20 con 0x7f3928107690 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.308+0000 7f391f7fe700 1 -- 172.21.15.92:0/1327464033 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f392001d470 con 0x7f3928107690 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.309+0000 7f391f7fe700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39140a2920 0x7f39140a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.309+0000 7f392e079700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39140a2920 0x7f39140a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.309+0000 7f391f7fe700 1 -- 172.21.15.92:0/1327464033 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f39200cfc50 con 0x7f3928107690 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.309+0000 7f391f7fe700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] conn(0x7f39140ac100 0x7f39140ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.309+0000 7f391f7fe700 1 -- 172.21.15.92:0/1327464033 --> [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3920029e70 con 0x7f39140ac100 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.309+0000 7f391f7fe700 1 -- 172.21.15.92:0/1327464033 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f39200d0130 con 0x7f3928107690 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.309+0000 7f392e079700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39140a2920 0x7f39140a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3928199a50 tx=0x7f3924009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:20.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.309+0000 7f392d878700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] conn(0x7f39140ac100 0x7f39140ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.310+0000 7f392d878700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] conn(0x7f39140ac100 0x7f39140ae520 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:20.317 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.311+0000 7f391f7fe700 1 -- 172.21.15.92:0/1327464033 <== osd.8 v2:172.21.15.159:6816/3217714611 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3920029e70 con 0x7f39140ac100 2024-03-19T01:51:20.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.334+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 --> [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3928062ea0 con 0x7f39140ac100 2024-03-19T01:51:20.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.334+0000 7f391f7fe700 1 -- 172.21.15.92:0/1327464033 <== osd.8 v2:172.21.15.159:6816/3217714611 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3928062ea0 con 0x7f39140ac100 2024-03-19T01:51:20.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 >> [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] conn(0x7f39140ac100 msgr2=0x7f39140ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] conn(0x7f39140ac100 0x7f39140ae520 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39140a2920 msgr2=0x7f39140a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39140a2920 0x7f39140a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3928199a50 tx=0x7f3924009260 comp rx=0 tx=0).stop 2024-03-19T01:51:20.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3928107690 msgr2=0x7f3928198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.335 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 --2- 172.21.15.92:0/1327464033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3928107690 0x7f3928198910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3920006070 tx=0x7f3920007a20 comp rx=0 tx=0).stop 2024-03-19T01:51:20.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f392e87a700 1 -- 172.21.15.92:0/1327464033 reap_dead start 2024-03-19T01:51:20.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 shutdown_connections 2024-03-19T01:51:20.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 >> 172.21.15.92:0/1327464033 conn(0x7f3928077260 msgr2=0x7f39280ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:20.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 shutdown_connections 2024-03-19T01:51:20.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.335+0000 7f39302dd700 1 -- 172.21.15.92:0/1327464033 wait complete. 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.519+0000 7ff723c28700 1 -- 172.21.15.92:0/3848401267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff71c10ca00 msgr2=0x7ff71c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.519+0000 7ff723c28700 1 --2- 172.21.15.92:0/3848401267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff71c10ca00 0x7ff71c10ee30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff71400b0a0 tx=0x7ff71400b3b0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.519+0000 7ff723c28700 1 -- 172.21.15.92:0/3848401267 shutdown_connections 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.519+0000 7ff723c28700 1 --2- 172.21.15.92:0/3848401267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff71c10ca00 0x7ff71c10ee30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.519+0000 7ff723c28700 1 --2- 172.21.15.92:0/3848401267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff71c103620 0x7ff71c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.519+0000 7ff723c28700 1 --2- 172.21.15.92:0/3848401267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c103f40 0x7ff71c10c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.519+0000 7ff723c28700 1 -- 172.21.15.92:0/3848401267 >> 172.21.15.92:0/3848401267 conn(0x7ff71c078340 msgr2=0x7ff71c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.519+0000 7ff723c28700 1 -- 172.21.15.92:0/3848401267 shutdown_connections 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.519+0000 7ff723c28700 1 -- 172.21.15.92:0/3848401267 wait complete. 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff723c28700 1 Processor -- start 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff723c28700 1 -- start start 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff723c28700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff71c103620 0x7ff71c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff723c28700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c103f40 0x7ff71c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff723c28700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff71c10ca00 0x7ff71c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff723c28700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1126d0 con 0x7ff71c103620 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff723c28700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c112550 con 0x7ff71c103f40 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff723c28700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c112850 con 0x7ff71c10ca00 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff7219c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff71c103620 0x7ff71c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff7219c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff71c103620 0x7ff71c19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58334/0 (socket says 172.21.15.92:58334) 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff7219c4700 1 -- 172.21.15.92:0/1034170722 learned_addr learned my addr 172.21.15.92:0/1034170722 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff7221c5700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff71c10ca00 0x7ff71c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.520+0000 7ff7211c3700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c103f40 0x7ff71c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff7219c4700 1 -- 172.21.15.92:0/1034170722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff71c10ca00 msgr2=0x7ff71c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff7219c4700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff71c10ca00 0x7ff71c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff7219c4700 1 -- 172.21.15.92:0/1034170722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c103f40 msgr2=0x7ff71c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff7219c4700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff71c103f40 0x7ff71c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff7219c4700 1 -- 172.21.15.92:0/1034170722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c1990d0 con 0x7ff71c103620 2024-03-19T01:51:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff7219c4700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff71c103620 0x7ff71c19e760 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff71800e9c0 tx=0x7ff71800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff712ffd700 1 -- 172.21.15.92:0/1034170722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71800cbd0 con 0x7ff71c103620 2024-03-19T01:51:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff712ffd700 1 -- 172.21.15.92:0/1034170722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71800ee60 con 0x7ff71c103620 2024-03-19T01:51:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c1993c0 con 0x7ff71c103620 2024-03-19T01:51:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff712ffd700 1 -- 172.21.15.92:0/1034170722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718005630 con 0x7ff71c103620 2024-03-19T01:51:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.521+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c1a5630 con 0x7ff71c103620 2024-03-19T01:51:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.522+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff71c0fec70 con 0x7ff71c103620 2024-03-19T01:51:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.524+0000 7ff712ffd700 1 -- 172.21.15.92:0/1034170722 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff718004750 con 0x7ff71c103620 2024-03-19T01:51:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.524+0000 7ff712ffd700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7080a2920 0x7ff7080a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.524+0000 7ff712ffd700 1 -- 172.21.15.92:0/1034170722 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff7180cf0e0 con 0x7ff71c103620 2024-03-19T01:51:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.524+0000 7ff7211c3700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7080a2920 0x7ff7080a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.524+0000 7ff712ffd700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] conn(0x7ff7080ac100 0x7ff7080ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.525+0000 7ff712ffd700 1 -- 172.21.15.92:0/1034170722 --> [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff71801fe50 con 0x7ff7080ac100 2024-03-19T01:51:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.525+0000 7ff712ffd700 1 -- 172.21.15.92:0/1034170722 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff7180cf5c0 con 0x7ff71c103620 2024-03-19T01:51:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.525+0000 7ff7221c5700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] conn(0x7ff7080ac100 0x7ff7080ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.525+0000 7ff7221c5700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] conn(0x7ff7080ac100 0x7ff7080ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.525+0000 7ff7211c3700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7080a2920 0x7ff7080a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff71c10e550 tx=0x7ff70c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.526+0000 7ff712ffd700 1 -- 172.21.15.92:0/1034170722 <== osd.9 v2:172.21.15.92:6826/4029500126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff71801fe50 con 0x7ff7080ac100 2024-03-19T01:51:20.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.548+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 --> [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff71c06b250 con 0x7ff7080ac100 2024-03-19T01:51:20.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff712ffd700 1 -- 172.21.15.92:0/1034170722 <== osd.9 v2:172.21.15.92:6826/4029500126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff71c06b250 con 0x7ff7080ac100 2024-03-19T01:51:20.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] conn(0x7ff7080ac100 msgr2=0x7ff7080ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] conn(0x7ff7080ac100 0x7ff7080ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7080a2920 msgr2=0x7ff7080a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7080a2920 0x7ff7080a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff71c10e550 tx=0x7ff70c009260 comp rx=0 tx=0).stop 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff71c103620 msgr2=0x7ff71c19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 --2- 172.21.15.92:0/1034170722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff71c103620 0x7ff71c19e760 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff71800e9c0 tx=0x7ff71800ecd0 comp rx=0 tx=0).stop 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff7221c5700 1 -- 172.21.15.92:0/1034170722 reap_dead start 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 shutdown_connections 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 >> 172.21.15.92:0/1034170722 conn(0x7ff71c078340 msgr2=0x7ff71c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 shutdown_connections 2024-03-19T01:51:20.550 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:20.549+0000 7ff723c28700 1 -- 172.21.15.92:0/1034170722 wait complete. 2024-03-19T01:51:20.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:20 smithi092 ceph-mon[62574]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-19T01:51:20.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:20 smithi144 ceph-mon[67199]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-19T01:51:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:20 smithi159 ceph-mon[67110]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-19T01:51:21.940 INFO:teuthology.orchestra.run.smithi092.stdout:253403070499 2024-03-19T01:51:21.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-19T01:51:22.606 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:22 smithi092 ceph-mon[62574]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-19T01:51:22.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:22 smithi144 ceph-mon[67199]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-19T01:51:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:22 smithi159 ceph-mon[67110]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-19T01:51:23.329 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.539+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/2081843754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48102390 msgr2=0x7f8e48102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.539+0000 7f8e4e9a9700 1 --2- 172.21.15.92:0/2081843754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48102390 0x7f8e48102810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009990 tx=0x7f8e38009ca0 comp rx=0 tx=0).stop 2024-03-19T01:51:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.539+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/2081843754 shutdown_connections 2024-03-19T01:51:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.539+0000 7f8e4e9a9700 1 --2- 172.21.15.92:0/2081843754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e48102d50 0x7f8e4810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.539+0000 7f8e4e9a9700 1 --2- 172.21.15.92:0/2081843754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e48108470 0x7f8e48108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.539+0000 7f8e4e9a9700 1 --2- 172.21.15.92:0/2081843754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48102390 0x7f8e48102810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.539+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/2081843754 >> 172.21.15.92:0/2081843754 conn(0x7f8e480793d0 msgr2=0x7f8e480797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.539+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/2081843754 shutdown_connections 2024-03-19T01:51:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.539+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/2081843754 wait complete. 2024-03-19T01:51:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.540+0000 7f8e4e9a9700 1 Processor -- start 2024-03-19T01:51:24.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.540+0000 7f8e4e9a9700 1 -- start start 2024-03-19T01:51:24.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.540+0000 7f8e4e9a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e48102390 0x7f8e48074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:24.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.540+0000 7f8e4e9a9700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e48102d50 0x7f8e48074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:24.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.540+0000 7f8e4e9a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48108470 0x7f8e48075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.540+0000 7f8e4e9a9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e481125d0 con 0x7f8e48108470 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.540+0000 7f8e4e9a9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e48112450 con 0x7f8e48102390 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.540+0000 7f8e4e9a9700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8e48112750 con 0x7f8e48102d50 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4cf46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48108470 0x7f8e48075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e477fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e48102d50 0x7f8e48074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4cf46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48108470 0x7f8e48075090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58860/0 (socket says 172.21.15.92:58860) 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4cf46700 1 -- 172.21.15.92:0/4272798505 learned_addr learned my addr 172.21.15.92:0/4272798505 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e47fff700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e48102390 0x7f8e48074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4cf46700 1 -- 172.21.15.92:0/4272798505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e48102d50 msgr2=0x7f8e48074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4cf46700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e48102d50 0x7f8e48074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4cf46700 1 -- 172.21.15.92:0/4272798505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e48102390 msgr2=0x7f8e48074610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4cf46700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e48102390 0x7f8e48074610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4cf46700 1 -- 172.21.15.92:0/4272798505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e481a6fb0 con 0x7f8e48108470 2024-03-19T01:51:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4cf46700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48108470 0x7f8e48075090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000ef30 tx=0x7f8e4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e457fa700 1 -- 172.21.15.92:0/4272798505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e400092e0 con 0x7f8e48108470 2024-03-19T01:51:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e457fa700 1 -- 172.21.15.92:0/4272798505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e40004bb0 con 0x7f8e48108470 2024-03-19T01:51:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e481a7240 con 0x7f8e48108470 2024-03-19T01:51:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e457fa700 1 -- 172.21.15.92:0/4272798505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40004280 con 0x7f8e48108470 2024-03-19T01:51:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.541+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e481a77a0 con 0x7f8e48108470 2024-03-19T01:51:24.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.542+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e48062a20 con 0x7f8e48108470 2024-03-19T01:51:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.543+0000 7f8e457fa700 1 -- 172.21.15.92:0/4272798505 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f8e4001b030 con 0x7f8e48108470 2024-03-19T01:51:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.544+0000 7f8e457fa700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8e300a29a0 0x7f8e300a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.544+0000 7f8e47fff700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8e300a29a0 0x7f8e300a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.544+0000 7f8e457fa700 1 -- 172.21.15.92:0/4272798505 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8e400ce430 con 0x7f8e48108470 2024-03-19T01:51:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.544+0000 7f8e457fa700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] conn(0x7f8e300ac180 0x7f8e300ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.544+0000 7f8e457fa700 1 -- 172.21.15.92:0/4272798505 --> [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e300aec50 con 0x7f8e300ac180 2024-03-19T01:51:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.544+0000 7f8e457fa700 1 -- 172.21.15.92:0/4272798505 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8e400d5050 con 0x7f8e48108470 2024-03-19T01:51:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.544+0000 7f8e477fe700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] conn(0x7f8e300ac180 0x7f8e300ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:24.545 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.544+0000 7f8e47fff700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8e300a29a0 0x7f8e300a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009be0 tx=0x7f8e3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:24.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.545+0000 7f8e477fe700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] conn(0x7f8e300ac180 0x7f8e300ae5a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:24.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.545+0000 7f8e457fa700 1 -- 172.21.15.92:0/4272798505 <== osd.5 v2:172.21.15.159:6808/3359154626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e300aec50 con 0x7f8e300ac180 2024-03-19T01:51:24.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.563+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 --> [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e48062ed0 con 0x7f8e300ac180 2024-03-19T01:51:24.563 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.563+0000 7f8e457fa700 1 -- 172.21.15.92:0/4272798505 <== osd.5 v2:172.21.15.159:6808/3359154626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8e48062ed0 con 0x7f8e300ac180 2024-03-19T01:51:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 >> [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] conn(0x7f8e300ac180 msgr2=0x7f8e300ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] conn(0x7f8e300ac180 0x7f8e300ae5a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8e300a29a0 msgr2=0x7f8e300a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8e300a29a0 0x7f8e300a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009be0 tx=0x7f8e3c009260 comp rx=0 tx=0).stop 2024-03-19T01:51:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48108470 msgr2=0x7f8e48075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 --2- 172.21.15.92:0/4272798505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e48108470 0x7f8e48075090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000ef30 tx=0x7f8e4000c3a0 comp rx=0 tx=0).stop 2024-03-19T01:51:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4cf46700 1 -- 172.21.15.92:0/4272798505 reap_dead start 2024-03-19T01:51:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 shutdown_connections 2024-03-19T01:51:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 >> 172.21.15.92:0/4272798505 conn(0x7f8e480793d0 msgr2=0x7f8e48101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 shutdown_connections 2024-03-19T01:51:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:24.564+0000 7f8e4e9a9700 1 -- 172.21.15.92:0/4272798505 wait complete. 2024-03-19T01:51:24.797 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:24 smithi092 ceph-mon[62574]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-19T01:51:24.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:24 smithi144 ceph-mon[67199]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-19T01:51:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:24 smithi159 ceph-mon[67110]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.867+0000 7ff374da5700 1 -- 172.21.15.92:0/3834848860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370107620 msgr2=0x7ff37010f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.867+0000 7ff374da5700 1 --2- 172.21.15.92:0/3834848860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370107620 0x7ff37010f1d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff36800b0a0 tx=0x7ff36800b3b0 comp rx=0 tx=0).stop 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.867+0000 7ff374da5700 1 -- 172.21.15.92:0/3834848860 shutdown_connections 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.867+0000 7ff374da5700 1 --2- 172.21.15.92:0/3834848860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370107620 0x7ff37010f1d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.867+0000 7ff374da5700 1 --2- 172.21.15.92:0/3834848860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff37006aab0 0x7ff37006ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.867+0000 7ff374da5700 1 --2- 172.21.15.92:0/3834848860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37006b460 0x7ff370106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.867+0000 7ff374da5700 1 -- 172.21.15.92:0/3834848860 >> 172.21.15.92:0/3834848860 conn(0x7ff370077260 msgr2=0x7ff370077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.868+0000 7ff374da5700 1 -- 172.21.15.92:0/3834848860 shutdown_connections 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.868+0000 7ff374da5700 1 -- 172.21.15.92:0/3834848860 wait complete. 2024-03-19T01:51:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.868+0000 7ff374da5700 1 Processor -- start 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff374da5700 1 -- start start 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff374da5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37006aab0 0x7ff37019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff374da5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff37006b460 0x7ff37019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff374da5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370107620 0x7ff3701987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff374da5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701125e0 con 0x7ff370107620 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff374da5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff370112460 con 0x7ff37006aab0 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff374da5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff370112760 con 0x7ff37006b460 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370107620 0x7ff3701987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36dd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff37006b460 0x7ff37019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37006aab0 0x7ff37019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37006aab0 0x7ff37019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:34328/0 (socket says 172.21.15.92:34328) 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36dd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff37006b460 0x7ff37019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:40134/0 (socket says 172.21.15.92:40134) 2024-03-19T01:51:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36e59c700 1 -- 172.21.15.92:0/3808065306 learned_addr learned my addr 172.21.15.92:0/3808065306 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36ed9d700 1 -- 172.21.15.92:0/3808065306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff37006b460 msgr2=0x7ff37019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36ed9d700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff37006b460 0x7ff37019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36ed9d700 1 -- 172.21.15.92:0/3808065306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37006aab0 msgr2=0x7ff37019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36ed9d700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37006aab0 0x7ff37019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36ed9d700 1 -- 172.21.15.92:0/3808065306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff370198f80 con 0x7ff370107620 2024-03-19T01:51:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36dd9b700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff37006b460 0x7ff37019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.869+0000 7ff36e59c700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff37006aab0 0x7ff37019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.870+0000 7ff36ed9d700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370107620 0x7ff3701987a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff36800b0a0 tx=0x7ff368004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.870+0000 7ff35f7fe700 1 -- 172.21.15.92:0/3808065306 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368021070 con 0x7ff370107620 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.870+0000 7ff35f7fe700 1 -- 172.21.15.92:0/3808065306 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff368007b80 con 0x7ff370107620 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.870+0000 7ff35f7fe700 1 -- 172.21.15.92:0/3808065306 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368005040 con 0x7ff370107620 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.870+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff370199210 con 0x7ff370107620 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.870+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3701a5480 con 0x7ff370107620 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.871+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff3700ffc10 con 0x7ff370107620 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.872+0000 7ff35f7fe700 1 -- 172.21.15.92:0/3808065306 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff368024070 con 0x7ff370107620 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.873+0000 7ff35f7fe700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3580a29a0 0x7ff3580a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.873+0000 7ff36e59c700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3580a29a0 0x7ff3580a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.873+0000 7ff35f7fe700 1 -- 172.21.15.92:0/3808065306 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff36800f020 con 0x7ff370107620 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.873+0000 7ff35f7fe700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] conn(0x7ff3580ac180 0x7ff3580ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.873+0000 7ff35f7fe700 1 -- 172.21.15.92:0/3808065306 --> [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3580aec50 con 0x7ff3580ac180 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.873+0000 7ff35f7fe700 1 -- 172.21.15.92:0/3808065306 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff3680d03a0 con 0x7ff370107620 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.873+0000 7ff36e59c700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3580a29a0 0x7ff3580a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff360011590 tx=0x7ff360009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.873+0000 7ff36dd9b700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] conn(0x7ff3580ac180 0x7ff3580ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.874+0000 7ff36dd9b700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] conn(0x7ff3580ac180 0x7ff3580ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.874+0000 7ff35f7fe700 1 -- 172.21.15.92:0/3808065306 <== osd.7 v2:172.21.15.144:6816/2720356070 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3580aec50 con 0x7ff3580ac180 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.891+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 --> [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff370103a20 con 0x7ff3580ac180 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.891+0000 7ff35f7fe700 1 -- 172.21.15.92:0/3808065306 <== osd.7 v2:172.21.15.144:6816/2720356070 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff370103a20 con 0x7ff3580ac180 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 >> [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] conn(0x7ff3580ac180 msgr2=0x7ff3580ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] conn(0x7ff3580ac180 0x7ff3580ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3580a29a0 msgr2=0x7ff3580a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff3580a29a0 0x7ff3580a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff360011590 tx=0x7ff360009260 comp rx=0 tx=0).stop 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370107620 msgr2=0x7ff3701987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 --2- 172.21.15.92:0/3808065306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370107620 0x7ff3701987a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff36800b0a0 tx=0x7ff368004d00 comp rx=0 tx=0).stop 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff36ed9d700 1 -- 172.21.15.92:0/3808065306 reap_dead start 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 shutdown_connections 2024-03-19T01:51:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 >> 172.21.15.92:0/3808065306 conn(0x7ff370077260 msgr2=0x7ff3700ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 shutdown_connections 2024-03-19T01:51:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:25.892+0000 7ff374da5700 1 -- 172.21.15.92:0/3808065306 wait complete. 2024-03-19T01:51:26.664 INFO:teuthology.orchestra.run.smithi092.stdout:141733920824 2024-03-19T01:51:26.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-19T01:51:26.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:26 smithi092 ceph-mon[62574]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-19T01:51:26.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:26 smithi144 ceph-mon[67199]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-19T01:51:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:26 smithi159 ceph-mon[67110]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-19T01:51:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.960+0000 7f178b0d5700 1 -- 172.21.15.92:0/227451659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17841014a0 msgr2=0x7f1784101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.960+0000 7f178b0d5700 1 --2- 172.21.15.92:0/227451659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17841014a0 0x7f1784101880 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f177c00b0d0 tx=0x7f177c00b3e0 comp rx=0 tx=0).stop 2024-03-19T01:51:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.960+0000 7f178b0d5700 1 -- 172.21.15.92:0/227451659 shutdown_connections 2024-03-19T01:51:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.960+0000 7f178b0d5700 1 --2- 172.21.15.92:0/227451659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17841064f0 0x7f178410e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.960+0000 7f178b0d5700 1 --2- 172.21.15.92:0/227451659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17841014a0 0x7f1784101880 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.960+0000 7f178b0d5700 1 --2- 172.21.15.92:0/227451659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1784101e50 0x7f1784105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.960+0000 7f178b0d5700 1 -- 172.21.15.92:0/227451659 >> 172.21.15.92:0/227451659 conn(0x7f1784077260 msgr2=0x7f1784077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.960+0000 7f178b0d5700 1 -- 172.21.15.92:0/227451659 shutdown_connections 2024-03-19T01:51:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.960+0000 7f178b0d5700 1 -- 172.21.15.92:0/227451659 wait complete. 2024-03-19T01:51:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.961+0000 7f178b0d5700 1 Processor -- start 2024-03-19T01:51:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.961+0000 7f178b0d5700 1 -- start start 2024-03-19T01:51:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.961+0000 7f178b0d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17841014a0 0x7f178419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.961+0000 7f178b0d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1784101e50 0x7f178419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.961+0000 7f178b0d5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17841064f0 0x7f1784197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.961+0000 7f178b0d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1784111500 con 0x7f1784101e50 2024-03-19T01:51:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.961+0000 7f178b0d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1784111380 con 0x7f17841014a0 2024-03-19T01:51:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.961+0000 7f178b0d5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1784111680 con 0x7f17841064f0 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1783fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1784101e50 0x7f178419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1783fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1784101e50 0x7f178419dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58898/0 (socket says 172.21.15.92:58898) 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1783fff700 1 -- 172.21.15.92:0/3326147014 learned_addr learned my addr 172.21.15.92:0/3326147014 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1788e71700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17841014a0 0x7f178419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1789672700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17841064f0 0x7f1784197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1783fff700 1 -- 172.21.15.92:0/3326147014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17841064f0 msgr2=0x7f1784197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1783fff700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17841064f0 0x7f1784197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1783fff700 1 -- 172.21.15.92:0/3326147014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17841014a0 msgr2=0x7f178419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1783fff700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17841014a0 0x7f178419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1783fff700 1 -- 172.21.15.92:0/3326147014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1784198000 con 0x7f1784101e50 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1783fff700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1784101e50 0x7f178419dbe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f177400e970 tx=0x7f177400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1781ffb700 1 -- 172.21.15.92:0/3326147014 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774004a30 con 0x7f1784101e50 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17841982f0 con 0x7f1784101e50 2024-03-19T01:51:26.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1781ffb700 1 -- 172.21.15.92:0/3326147014 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f177400ccc0 con 0x7f1784101e50 2024-03-19T01:51:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f1781ffb700 1 -- 172.21.15.92:0/3326147014 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774005650 con 0x7f1784101e50 2024-03-19T01:51:26.965 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.962+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841a4500 con 0x7f1784101e50 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.963+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1784062a60 con 0x7f1784101e50 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.965+0000 7f1781ffb700 1 -- 172.21.15.92:0/3326147014 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f1774004bd0 con 0x7f1784101e50 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.965+0000 7f1781ffb700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17700a2920 0x7f17700a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.965+0000 7f1788e71700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17700a2920 0x7f17700a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.965+0000 7f1781ffb700 1 -- 172.21.15.92:0/3326147014 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f17740cf0b0 con 0x7f1784101e50 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.966+0000 7f1781ffb700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] conn(0x7f17700ac100 0x7f17700ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.966+0000 7f1781ffb700 1 -- 172.21.15.92:0/3326147014 --> [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1774015e80 con 0x7f17700ac100 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.966+0000 7f1781ffb700 1 -- 172.21.15.92:0/3326147014 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f17740cf590 con 0x7f1784101e50 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.966+0000 7f1789672700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] conn(0x7f17700ac100 0x7f17700ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.966+0000 7f1788e71700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17700a2920 0x7f17700a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f177c009fd0 tx=0x7f177c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:26.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.966+0000 7f1789672700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] conn(0x7f17700ac100 0x7f17700ae520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:26.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.967+0000 7f1781ffb700 1 -- 172.21.15.92:0/3326147014 <== osd.0 v2:172.21.15.92:6802/2208105319 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1774015e80 con 0x7f17700ac100 2024-03-19T01:51:26.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.984+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 --> [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1784062e60 con 0x7f17700ac100 2024-03-19T01:51:26.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f1781ffb700 1 -- 172.21.15.92:0/3326147014 <== osd.0 v2:172.21.15.92:6802/2208105319 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1784062e60 con 0x7f17700ac100 2024-03-19T01:51:26.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] conn(0x7f17700ac100 msgr2=0x7f17700ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] conn(0x7f17700ac100 0x7f17700ae520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17700a2920 msgr2=0x7f17700a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17700a2920 0x7f17700a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f177c009fd0 tx=0x7f177c00b040 comp rx=0 tx=0).stop 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1784101e50 msgr2=0x7f178419dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 --2- 172.21.15.92:0/3326147014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1784101e50 0x7f178419dbe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f177400e970 tx=0x7f177400ec80 comp rx=0 tx=0).stop 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f1789672700 1 -- 172.21.15.92:0/3326147014 reap_dead start 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 shutdown_connections 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 >> 172.21.15.92:0/3326147014 conn(0x7f1784077260 msgr2=0x7f1784100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 shutdown_connections 2024-03-19T01:51:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:26.985+0000 7f178b0d5700 1 -- 172.21.15.92:0/3326147014 wait complete. 2024-03-19T01:51:27.428 INFO:teuthology.orchestra.run.smithi092.stdout:287762808861 2024-03-19T01:51:27.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-19T01:51:28.075 INFO:teuthology.orchestra.run.smithi092.stdout:163208757300 2024-03-19T01:51:28.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-19T01:51:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:28 smithi092 ceph-mon[62574]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-19T01:51:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:28 smithi144 ceph-mon[67199]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-19T01:51:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:28 smithi159 ceph-mon[67110]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-19T01:51:29.501 INFO:teuthology.orchestra.run.smithi092.stdout:219043332138 2024-03-19T01:51:29.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-19T01:51:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.356+0000 7f71ddc13700 1 -- 172.21.15.92:0/3718912786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8107380 msgr2=0x7f71d8107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.356+0000 7f71ddc13700 1 --2- 172.21.15.92:0/3718912786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8107380 0x7f71d8107760 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009990 tx=0x7f71c8009ca0 comp rx=0 tx=0).stop 2024-03-19T01:51:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.356+0000 7f71ddc13700 1 -- 172.21.15.92:0/3718912786 shutdown_connections 2024-03-19T01:51:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.356+0000 7f71ddc13700 1 --2- 172.21.15.92:0/3718912786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d81012c0 0x7f71d810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.356+0000 7f71ddc13700 1 --2- 172.21.15.92:0/3718912786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8107380 0x7f71d8107760 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.356+0000 7f71ddc13700 1 --2- 172.21.15.92:0/3718912786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71d8100900 0x7f71d8100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.356+0000 7f71ddc13700 1 -- 172.21.15.92:0/3718912786 >> 172.21.15.92:0/3718912786 conn(0x7f71d8077810 msgr2=0x7f71d80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.356+0000 7f71ddc13700 1 -- 172.21.15.92:0/3718912786 shutdown_connections 2024-03-19T01:51:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.356+0000 7f71ddc13700 1 -- 172.21.15.92:0/3718912786 wait complete. 2024-03-19T01:51:30.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71ddc13700 1 Processor -- start 2024-03-19T01:51:30.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71ddc13700 1 -- start start 2024-03-19T01:51:30.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71ddc13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8100900 0x7f71d819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:30.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71ddc13700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71d81012c0 0x7f71d819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:30.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71ddc13700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d8107380 0x7f71d81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:30.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71ddc13700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8112760 con 0x7f71d8100900 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71ddc13700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71d81125e0 con 0x7f71d8107380 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71ddc13700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f71d81128e0 con 0x7f71d81012c0 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8100900 0x7f71d819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8100900 0x7f71d819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58924/0 (socket says 172.21.15.92:58924) 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71d77fe700 1 -- 172.21.15.92:0/3321467479 learned_addr learned my addr 172.21.15.92:0/3321467479 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71d6ffd700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71d81012c0 0x7f71d819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.357+0000 7f71d7fff700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d8107380 0x7f71d81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71d77fe700 1 -- 172.21.15.92:0/3321467479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71d81012c0 msgr2=0x7f71d819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71d77fe700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71d81012c0 0x7f71d819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71d77fe700 1 -- 172.21.15.92:0/3321467479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d8107380 msgr2=0x7f71d81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71d77fe700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71d8107380 0x7f71d81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71d77fe700 1 -- 172.21.15.92:0/3321467479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d8199140 con 0x7f71d8100900 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71d77fe700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8100900 0x7f71d819e820 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f71c8000c00 tx=0x7f71c800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:30.360 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71d4ff9700 1 -- 172.21.15.92:0/3321467479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8027070 con 0x7f71d8100900 2024-03-19T01:51:30.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71d4ff9700 1 -- 172.21.15.92:0/3321467479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c8003d70 con 0x7f71d8100900 2024-03-19T01:51:30.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d81993d0 con 0x7f71d8100900 2024-03-19T01:51:30.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71d4ff9700 1 -- 172.21.15.92:0/3321467479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800f460 con 0x7f71d8100900 2024-03-19T01:51:30.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.358+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d81a56f0 con 0x7f71d8100900 2024-03-19T01:51:30.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.359+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71d8062a20 con 0x7f71d8100900 2024-03-19T01:51:30.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.360+0000 7f71d4ff9700 1 -- 172.21.15.92:0/3321467479 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f71c802a070 con 0x7f71d8100900 2024-03-19T01:51:30.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.361+0000 7f71d4ff9700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f71c00a2920 0x7f71c00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:30.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.361+0000 7f71d6ffd700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f71c00a2920 0x7f71c00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:30.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.361+0000 7f71d4ff9700 1 -- 172.21.15.92:0/3321467479 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f71c80cf7c0 con 0x7f71d8100900 2024-03-19T01:51:30.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.361+0000 7f71d4ff9700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] conn(0x7f71c00ac100 0x7f71c00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:30.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.361+0000 7f71d4ff9700 1 -- 172.21.15.92:0/3321467479 --> [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71c800eea0 con 0x7f71c00ac100 2024-03-19T01:51:30.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.361+0000 7f71d4ff9700 1 -- 172.21.15.92:0/3321467479 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f71c80d42c0 con 0x7f71d8100900 2024-03-19T01:51:30.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.361+0000 7f71d6ffd700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f71c00a2920 0x7f71c00a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f71d80ffdb0 tx=0x7f71cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:30.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.361+0000 7f71d7fff700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] conn(0x7f71c00ac100 0x7f71c00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:30.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.362+0000 7f71d7fff700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] conn(0x7f71c00ac100 0x7f71c00ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:30.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.362+0000 7f71d4ff9700 1 -- 172.21.15.92:0/3321467479 <== osd.3 v2:172.21.15.92:6810/1878896189 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71c800eea0 con 0x7f71c00ac100 2024-03-19T01:51:30.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 --> [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71d8062e20 con 0x7f71c00ac100 2024-03-19T01:51:30.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71d4ff9700 1 -- 172.21.15.92:0/3321467479 <== osd.3 v2:172.21.15.92:6810/1878896189 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f71d8062e20 con 0x7f71c00ac100 2024-03-19T01:51:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] conn(0x7f71c00ac100 msgr2=0x7f71c00ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71ddc13700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] conn(0x7f71c00ac100 0x7f71c00ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f71c00a2920 msgr2=0x7f71c00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71ddc13700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f71c00a2920 0x7f71c00a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f71d80ffdb0 tx=0x7f71cc009260 comp rx=0 tx=0).stop 2024-03-19T01:51:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8100900 msgr2=0x7f71d819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71ddc13700 1 --2- 172.21.15.92:0/3321467479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71d8100900 0x7f71d819e820 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f71c8000c00 tx=0x7f71c800bab0 comp rx=0 tx=0).stop 2024-03-19T01:51:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71d7fff700 1 -- 172.21.15.92:0/3321467479 reap_dead start 2024-03-19T01:51:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 shutdown_connections 2024-03-19T01:51:30.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.379+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 >> 172.21.15.92:0/3321467479 conn(0x7f71d8077810 msgr2=0x7f71d8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.380+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 shutdown_connections 2024-03-19T01:51:30.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:30.380+0000 7f71ddc13700 1 -- 172.21.15.92:0/3321467479 wait complete. 2024-03-19T01:51:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:30 smithi092 ceph-mon[62574]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-19T01:51:30.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:30 smithi144 ceph-mon[67199]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-19T01:51:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:30 smithi159 ceph-mon[67110]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-19T01:51:32.047 INFO:teuthology.orchestra.run.smithi092.stdout:34359738445 2024-03-19T01:51:32.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-19T01:51:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:32 smithi092 ceph-mon[62574]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:51:32.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:32 smithi144 ceph-mon[67199]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:51:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:32 smithi159 ceph-mon[67110]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:51:32.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 -- 172.21.15.92:0/1539421267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7841018c0 msgr2=0x7fe784107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:32.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 --2- 172.21.15.92:0/1539421267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7841018c0 0x7fe784107000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0099c0 tx=0x7fe76c009cd0 comp rx=0 tx=0).stop 2024-03-19T01:51:32.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 -- 172.21.15.92:0/1539421267 shutdown_connections 2024-03-19T01:51:32.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 --2- 172.21.15.92:0/1539421267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7841076e0 0x7fe78410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:32.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 --2- 172.21.15.92:0/1539421267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784100f10 0x7fe7841012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:32.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 --2- 172.21.15.92:0/1539421267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7841018c0 0x7fe784107000 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:32.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 -- 172.21.15.92:0/1539421267 >> 172.21.15.92:0/1539421267 conn(0x7fe784076590 msgr2=0x7fe7840769a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:32.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 -- 172.21.15.92:0/1539421267 shutdown_connections 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 -- 172.21.15.92:0/1539421267 wait complete. 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 Processor -- start 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.556+0000 7fe78a2ea700 1 -- start start 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe78a2ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784100f10 0x7fe78419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe78a2ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7841018c0 0x7fe78419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe78a2ea700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7841076e0 0x7fe784198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe78a2ea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe784112700 con 0x7fe7841018c0 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe78a2ea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe784112580 con 0x7fe784100f10 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe78a2ea700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe784112880 con 0x7fe7841076e0 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe7837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7841018c0 0x7fe78419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe783fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784100f10 0x7fe78419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe7837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7841018c0 0x7fe78419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58936/0 (socket says 172.21.15.92:58936) 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe7837fe700 1 -- 172.21.15.92:0/4057386998 learned_addr learned my addr 172.21.15.92:0/4057386998 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:32.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe788887700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7841076e0 0x7fe784198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:32.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe783fff700 1 -- 172.21.15.92:0/4057386998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7841076e0 msgr2=0x7fe784198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:32.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe783fff700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7841076e0 0x7fe784198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:32.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe783fff700 1 -- 172.21.15.92:0/4057386998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7841018c0 msgr2=0x7fe78419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:32.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe783fff700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7841018c0 0x7fe78419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:32.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe783fff700 1 -- 172.21.15.92:0/4057386998 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe784199180 con 0x7fe784100f10 2024-03-19T01:51:32.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe7837fe700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7841018c0 0x7fe78419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:32.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.557+0000 7fe783fff700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784100f10 0x7fe78419e810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe77400e9f0 tx=0x7fe77400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:32.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.558+0000 7fe7817fa700 1 -- 172.21.15.92:0/4057386998 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77400c9c0 con 0x7fe784100f10 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.558+0000 7fe7817fa700 1 -- 172.21.15.92:0/4057386998 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77400cb60 con 0x7fe784100f10 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.558+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe784199470 con 0x7fe784100f10 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.558+0000 7fe7817fa700 1 -- 172.21.15.92:0/4057386998 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774018680 con 0x7fe784100f10 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.558+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7841a56e0 con 0x7fe784100f10 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.558+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe784062a20 con 0x7fe784100f10 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.560+0000 7fe7817fa700 1 -- 172.21.15.92:0/4057386998 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fe774010430 con 0x7fe784100f10 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.560+0000 7fe7817fa700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe7700a29a0 0x7fe7700a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.560+0000 7fe7837fe700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe7700a29a0 0x7fe7700a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.560+0000 7fe7817fa700 1 -- 172.21.15.92:0/4057386998 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe7740cefd0 con 0x7fe784100f10 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.561+0000 7fe7817fa700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] conn(0x7fe7700ac180 0x7fe7700ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.561+0000 7fe7817fa700 1 -- 172.21.15.92:0/4057386998 --> [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe77401bec0 con 0x7fe7700ac180 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.561+0000 7fe7817fa700 1 -- 172.21.15.92:0/4057386998 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fe7740cf440 con 0x7fe784100f10 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.561+0000 7fe788887700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] conn(0x7fe7700ac180 0x7fe7700ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:32.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.561+0000 7fe7837fe700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe7700a29a0 0x7fe7700a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe7841007c0 tx=0x7fe76c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.561+0000 7fe788887700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] conn(0x7fe7700ac180 0x7fe7700ae5a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.562+0000 7fe7817fa700 1 -- 172.21.15.92:0/4057386998 <== osd.11 v2:172.21.15.159:6824/3616019688 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe77401bec0 con 0x7fe7700ac180 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.578+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 --> [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe784062e20 con 0x7fe7700ac180 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.578+0000 7fe7817fa700 1 -- 172.21.15.92:0/4057386998 <== osd.11 v2:172.21.15.159:6824/3616019688 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe784062e20 con 0x7fe7700ac180 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 >> [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] conn(0x7fe7700ac180 msgr2=0x7fe7700ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] conn(0x7fe7700ac180 0x7fe7700ae5a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe7700a29a0 msgr2=0x7fe7700a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe7700a29a0 0x7fe7700a4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe7841007c0 tx=0x7fe76c005fb0 comp rx=0 tx=0).stop 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784100f10 msgr2=0x7fe78419e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 --2- 172.21.15.92:0/4057386998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe784100f10 0x7fe78419e810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe77400e9f0 tx=0x7fe77400ed00 comp rx=0 tx=0).stop 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe788887700 1 -- 172.21.15.92:0/4057386998 reap_dead start 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 shutdown_connections 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 >> 172.21.15.92:0/4057386998 conn(0x7fe784076590 msgr2=0x7fe7840fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 shutdown_connections 2024-03-19T01:51:32.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:32.579+0000 7fe78a2ea700 1 -- 172.21.15.92:0/4057386998 wait complete. 2024-03-19T01:51:34.079 INFO:teuthology.orchestra.run.smithi092.stdout:343597383702 2024-03-19T01:51:34.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-19T01:51:34.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.401+0000 7f9524136700 1 -- 172.21.15.92:0/2844102339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f951c108340 msgr2=0x7f951c108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:34.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.401+0000 7f9524136700 1 --2- 172.21.15.92:0/2844102339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f951c108340 0x7f951c108720 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f95180099c0 tx=0x7f9518009cd0 comp rx=0 tx=0).stop 2024-03-19T01:51:34.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.401+0000 7f9524136700 1 -- 172.21.15.92:0/2844102339 shutdown_connections 2024-03-19T01:51:34.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.401+0000 7f9524136700 1 --2- 172.21.15.92:0/2844102339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f951c102d20 0x7f951c10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:34.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.401+0000 7f9524136700 1 --2- 172.21.15.92:0/2844102339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f951c108340 0x7f951c108720 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:34.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.401+0000 7f9524136700 1 --2- 172.21.15.92:0/2844102339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f951c102360 0x7f951c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:34.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.401+0000 7f9524136700 1 -- 172.21.15.92:0/2844102339 >> 172.21.15.92:0/2844102339 conn(0x7f951c079440 msgr2=0x7f951c079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:34.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.401+0000 7f9524136700 1 -- 172.21.15.92:0/2844102339 shutdown_connections 2024-03-19T01:51:34.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.401+0000 7f9524136700 1 -- 172.21.15.92:0/2844102339 wait complete. 2024-03-19T01:51:34.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9524136700 1 Processor -- start 2024-03-19T01:51:34.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9524136700 1 -- start start 2024-03-19T01:51:34.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9524136700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f951c102360 0x7f951c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9524136700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f951c102d20 0x7f951c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9524136700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f951c108340 0x7f951c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9524136700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f951c1125f0 con 0x7f951c102d20 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9524136700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f951c112470 con 0x7f951c108340 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9524136700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f951c112770 con 0x7f951c102360 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9521ed2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f951c102360 0x7f951c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9521ed2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f951c102360 0x7f951c19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:44382/0 (socket says 172.21.15.92:44382) 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9521ed2700 1 -- 172.21.15.92:0/4185288115 learned_addr learned my addr 172.21.15.92:0/4185288115 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f95216d1700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f951c102d20 0x7f951c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f95226d3700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f951c108340 0x7f951c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9521ed2700 1 -- 172.21.15.92:0/4185288115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f951c108340 msgr2=0x7f951c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9521ed2700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f951c108340 0x7f951c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9521ed2700 1 -- 172.21.15.92:0/4185288115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f951c102d20 msgr2=0x7f951c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.402+0000 7f9521ed2700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f951c102d20 0x7f951c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:34.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.403+0000 7f9521ed2700 1 -- 172.21.15.92:0/4185288115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c199010 con 0x7f951c102360 2024-03-19T01:51:34.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.403+0000 7f9521ed2700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f951c102360 0x7f951c19e6f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9518006010 tx=0x7f951800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:34.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.403+0000 7f9512ffd700 1 -- 172.21.15.92:0/4185288115 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518003680 con 0x7f951c102360 2024-03-19T01:51:34.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.403+0000 7f9512ffd700 1 -- 172.21.15.92:0/4185288115 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95180050d0 con 0x7f951c102360 2024-03-19T01:51:34.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.403+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951c1992a0 con 0x7f951c102360 2024-03-19T01:51:34.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.403+0000 7f9512ffd700 1 -- 172.21.15.92:0/4185288115 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518003db0 con 0x7f951c102360 2024-03-19T01:51:34.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.403+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f951c1a55c0 con 0x7f951c102360 2024-03-19T01:51:34.406 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.404+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f951c062a60 con 0x7f951c102360 2024-03-19T01:51:34.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.405+0000 7f9512ffd700 1 -- 172.21.15.92:0/4185288115 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9518005280 con 0x7f951c102360 2024-03-19T01:51:34.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.406+0000 7f9512ffd700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95080a2ac0 0x7f95080a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:34.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.406+0000 7f95216d1700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95080a2ac0 0x7f95080a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:34.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.406+0000 7f9512ffd700 1 -- 172.21.15.92:0/4185288115 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f95180cfbd0 con 0x7f951c102360 2024-03-19T01:51:34.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.406+0000 7f9512ffd700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] conn(0x7f95080ac2a0 0x7f95080ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:34.407 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.406+0000 7f9512ffd700 1 -- 172.21.15.92:0/4185288115 --> [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f951800fe50 con 0x7f95080ac2a0 2024-03-19T01:51:34.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.406+0000 7f9512ffd700 1 -- 172.21.15.92:0/4185288115 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f95180cff80 con 0x7f951c102360 2024-03-19T01:51:34.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.406+0000 7f95226d3700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] conn(0x7f95080ac2a0 0x7f95080ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:34.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.407+0000 7f95216d1700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95080a2ac0 0x7f95080a4f80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f951c079a90 tx=0x7f950c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:34.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.407+0000 7f95226d3700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] conn(0x7f95080ac2a0 0x7f95080ae6c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:34.408 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.408+0000 7f9512ffd700 1 -- 172.21.15.92:0/4185288115 <== osd.2 v2:172.21.15.159:6800/2365994247 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f951800fe50 con 0x7f95080ac2a0 2024-03-19T01:51:34.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.437+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 --> [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f951c062ee0 con 0x7f95080ac2a0 2024-03-19T01:51:34.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.437+0000 7f9512ffd700 1 -- 172.21.15.92:0/4185288115 <== osd.2 v2:172.21.15.159:6800/2365994247 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f951c062ee0 con 0x7f95080ac2a0 2024-03-19T01:51:34.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.438+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 >> [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] conn(0x7f95080ac2a0 msgr2=0x7f95080ae6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:34.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.438+0000 7f9524136700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] conn(0x7f95080ac2a0 0x7f95080ae6c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:34.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.438+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95080a2ac0 msgr2=0x7f95080a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:34.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.438+0000 7f9524136700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95080a2ac0 0x7f95080a4f80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f951c079a90 tx=0x7f950c009260 comp rx=0 tx=0).stop 2024-03-19T01:51:34.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.438+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f951c102360 msgr2=0x7f951c19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:34.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.438+0000 7f9524136700 1 --2- 172.21.15.92:0/4185288115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f951c102360 0x7f951c19e6f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9518006010 tx=0x7f951800f690 comp rx=0 tx=0).stop 2024-03-19T01:51:34.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.438+0000 7f95226d3700 1 -- 172.21.15.92:0/4185288115 reap_dead start 2024-03-19T01:51:34.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.438+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 shutdown_connections 2024-03-19T01:51:34.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.438+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 >> 172.21.15.92:0/4185288115 conn(0x7f951c079440 msgr2=0x7f951c101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:34.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.439+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 shutdown_connections 2024-03-19T01:51:34.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:34.439+0000 7f9524136700 1 -- 172.21.15.92:0/4185288115 wait complete. 2024-03-19T01:51:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:34 smithi092 ceph-mon[62574]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-03-19T01:51:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:34 smithi144 ceph-mon[67199]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-03-19T01:51:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:34 smithi159 ceph-mon[67110]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-03-19T01:51:35.119 INFO:teuthology.orchestra.run.smithi092.stdout:107374182462 2024-03-19T01:51:35.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-19T01:51:36.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.664+0000 7fb7b3c58700 1 -- 172.21.15.92:0/1159299181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ac101210 msgr2=0x7fb7ac101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.664+0000 7fb7b3c58700 1 --2- 172.21.15.92:0/1159299181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ac101210 0x7fb7ac101670 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009990 tx=0x7fb79c009ca0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.664+0000 7fb7b3c58700 1 -- 172.21.15.92:0/1159299181 shutdown_connections 2024-03-19T01:51:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.664+0000 7fb7b3c58700 1 --2- 172.21.15.92:0/1159299181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ac101bb0 0x7fb7ac10e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.664+0000 7fb7b3c58700 1 --2- 172.21.15.92:0/1159299181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7ac107250 0x7fb7ac107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.664+0000 7fb7b3c58700 1 --2- 172.21.15.92:0/1159299181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ac101210 0x7fb7ac101670 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.664+0000 7fb7b3c58700 1 -- 172.21.15.92:0/1159299181 >> 172.21.15.92:0/1159299181 conn(0x7fb7ac078140 msgr2=0x7fb7ac078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.664+0000 7fb7b3c58700 1 -- 172.21.15.92:0/1159299181 shutdown_connections 2024-03-19T01:51:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.664+0000 7fb7b3c58700 1 -- 172.21.15.92:0/1159299181 wait complete. 2024-03-19T01:51:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b3c58700 1 Processor -- start 2024-03-19T01:51:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b3c58700 1 -- start start 2024-03-19T01:51:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b3c58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ac101210 0x7fb7ac19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b3c58700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7ac101bb0 0x7fb7ac19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b3c58700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ac107250 0x7fb7ac1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b3c58700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac1114b0 con 0x7fb7ac101210 2024-03-19T01:51:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b3c58700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac111330 con 0x7fb7ac101bb0 2024-03-19T01:51:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b3c58700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac111630 con 0x7fb7ac107250 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b19f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ac101210 0x7fb7ac19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b19f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ac101210 0x7fb7ac19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33440/0 (socket says 172.21.15.92:33440) 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b19f4700 1 -- 172.21.15.92:0/2914294135 learned_addr learned my addr 172.21.15.92:0/2914294135 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b11f3700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7ac101bb0 0x7fb7ac19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.665+0000 7fb7b21f5700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ac107250 0x7fb7ac1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7b19f4700 1 -- 172.21.15.92:0/2914294135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ac107250 msgr2=0x7fb7ac1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7b19f4700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ac107250 0x7fb7ac1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7b19f4700 1 -- 172.21.15.92:0/2914294135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7ac101bb0 msgr2=0x7fb7ac19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7b19f4700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7ac101bb0 0x7fb7ac19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7b19f4700 1 -- 172.21.15.92:0/2914294135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac197fb0 con 0x7fb7ac101210 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7b19f4700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ac101210 0x7fb7ac19d650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c370 tx=0x7fb7a400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7a2ffd700 1 -- 172.21.15.92:0/2914294135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400cea0 con 0x7fb7ac101210 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7a2ffd700 1 -- 172.21.15.92:0/2914294135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a40090d0 con 0x7fb7ac101210 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7a2ffd700 1 -- 172.21.15.92:0/2914294135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4004770 con 0x7fb7ac101210 2024-03-19T01:51:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ac198240 con 0x7fb7ac101210 2024-03-19T01:51:36.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.666+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ac1a44b0 con 0x7fb7ac101210 2024-03-19T01:51:36.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.667+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7ac062aa0 con 0x7fb7ac101210 2024-03-19T01:51:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.668+0000 7fb7a2ffd700 1 -- 172.21.15.92:0/2914294135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb7a401a020 con 0x7fb7ac101210 2024-03-19T01:51:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.669+0000 7fb7a2ffd700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7980a29a0 0x7fb7980a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.669+0000 7fb7b11f3700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7980a29a0 0x7fb7980a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.669+0000 7fb7a2ffd700 1 -- 172.21.15.92:0/2914294135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb7a40ced20 con 0x7fb7ac101210 2024-03-19T01:51:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.669+0000 7fb7a2ffd700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] conn(0x7fb7980ac180 0x7fb7980ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.669+0000 7fb7a2ffd700 1 -- 172.21.15.92:0/2914294135 --> [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7a4008ec0 con 0x7fb7980ac180 2024-03-19T01:51:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.669+0000 7fb7a2ffd700 1 -- 172.21.15.92:0/2914294135 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb7a40cf200 con 0x7fb7ac101210 2024-03-19T01:51:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.669+0000 7fb7b21f5700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] conn(0x7fb7980ac180 0x7fb7980ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.670+0000 7fb7b11f3700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7980a29a0 0x7fb7980a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0098e0 tx=0x7fb79c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.670+0000 7fb7b21f5700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] conn(0x7fb7980ac180 0x7fb7980ae5a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.670+0000 7fb7a2ffd700 1 -- 172.21.15.92:0/2914294135 <== osd.6 v2:172.21.15.92:6818/1165350802 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7a4008ec0 con 0x7fb7980ac180 2024-03-19T01:51:36.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.686+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 --> [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7ac062ea0 con 0x7fb7980ac180 2024-03-19T01:51:36.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7a2ffd700 1 -- 172.21.15.92:0/2914294135 <== osd.6 v2:172.21.15.92:6818/1165350802 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb7ac062ea0 con 0x7fb7980ac180 2024-03-19T01:51:36.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] conn(0x7fb7980ac180 msgr2=0x7fb7980ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.687 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] conn(0x7fb7980ac180 0x7fb7980ae5a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7980a29a0 msgr2=0x7fb7980a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7980a29a0 0x7fb7980a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0098e0 tx=0x7fb79c005960 comp rx=0 tx=0).stop 2024-03-19T01:51:36.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ac101210 msgr2=0x7fb7ac19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 --2- 172.21.15.92:0/2914294135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ac101210 0x7fb7ac19d650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c370 tx=0x7fb7a400c680 comp rx=0 tx=0).stop 2024-03-19T01:51:36.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b21f5700 1 -- 172.21.15.92:0/2914294135 reap_dead start 2024-03-19T01:51:36.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 shutdown_connections 2024-03-19T01:51:36.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 >> 172.21.15.92:0/2914294135 conn(0x7fb7ac078140 msgr2=0x7fb7ac0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:36.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 shutdown_connections 2024-03-19T01:51:36.688 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.687+0000 7fb7b3c58700 1 -- 172.21.15.92:0/2914294135 wait complete. 2024-03-19T01:51:36.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.539+0000 7ff97815f700 1 -- 172.21.15.92:0/2321240850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9701064f0 msgr2=0x7ff97010e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.539+0000 7ff97815f700 1 --2- 172.21.15.92:0/2321240850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9701064f0 0x7ff97010e0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff96800b0d0 tx=0x7ff96800b3e0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.539+0000 7ff97815f700 1 -- 172.21.15.92:0/2321240850 shutdown_connections 2024-03-19T01:51:36.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.539+0000 7ff97815f700 1 --2- 172.21.15.92:0/2321240850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9701064f0 0x7ff97010e0c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.539+0000 7ff97815f700 1 --2- 172.21.15.92:0/2321240850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9701014a0 0x7ff970101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.539+0000 7ff97815f700 1 --2- 172.21.15.92:0/2321240850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff970101e50 0x7ff970105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.539+0000 7ff97815f700 1 -- 172.21.15.92:0/2321240850 >> 172.21.15.92:0/2321240850 conn(0x7ff970077260 msgr2=0x7ff970077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 -- 172.21.15.92:0/2321240850 shutdown_connections 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 -- 172.21.15.92:0/2321240850 wait complete. 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 Processor -- start 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 -- start start 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9701014a0 0x7ff97019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff970101e50 0x7ff97019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9701064f0 0x7ff9701988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff970111550 con 0x7ff9701064f0 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701113d0 con 0x7ff9701014a0 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.540+0000 7ff97815f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701116d0 con 0x7ff970101e50 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9766fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9701064f0 0x7ff9701988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9766fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9701064f0 0x7ff9701988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33422/0 (socket says 172.21.15.92:33422) 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9766fc700 1 -- 172.21.15.92:0/3962777672 learned_addr learned my addr 172.21.15.92:0/3962777672 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9756fa700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff970101e50 0x7ff97019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff975efb700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9701014a0 0x7ff97019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9766fc700 1 -- 172.21.15.92:0/3962777672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff970101e50 msgr2=0x7ff97019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9766fc700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff970101e50 0x7ff97019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9766fc700 1 -- 172.21.15.92:0/3962777672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9701014a0 msgr2=0x7ff97019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9766fc700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9701014a0 0x7ff97019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9766fc700 1 -- 172.21.15.92:0/3962777672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff970199000 con 0x7ff9701064f0 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff9766fc700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9701064f0 0x7ff9701988a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff96800d010 tx=0x7ff9680079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff966ffd700 1 -- 172.21.15.92:0/3962777672 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968003c00 con 0x7ff9701064f0 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff966ffd700 1 -- 172.21.15.92:0/3962777672 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff968022070 con 0x7ff9701064f0 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff970199290 con 0x7ff9701064f0 2024-03-19T01:51:36.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff966ffd700 1 -- 172.21.15.92:0/3962777672 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96801b6c0 con 0x7ff9701064f0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.541+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9701a55b0 con 0x7ff9701064f0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.542+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff970062aa0 con 0x7ff9701064f0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.543+0000 7ff966ffd700 1 -- 172.21.15.92:0/3962777672 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff968009ba0 con 0x7ff9701064f0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.544+0000 7ff966ffd700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff95c0a29a0 0x7ff95c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.544+0000 7ff975efb700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff95c0a29a0 0x7ff95c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.544+0000 7ff966ffd700 1 -- 172.21.15.92:0/3962777672 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff96800f020 con 0x7ff9701064f0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.544+0000 7ff966ffd700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] conn(0x7ff95c0ac180 0x7ff95c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.544+0000 7ff966ffd700 1 -- 172.21.15.92:0/3962777672 --> [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff95c0aec50 con 0x7ff95c0ac180 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.544+0000 7ff966ffd700 1 -- 172.21.15.92:0/3962777672 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff9680d6050 con 0x7ff9701064f0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.544+0000 7ff975efb700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff95c0a29a0 0x7ff95c0a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff9701999e0 tx=0x7ff96c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.544+0000 7ff9756fa700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] conn(0x7ff95c0ac180 0x7ff95c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.545+0000 7ff9756fa700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] conn(0x7ff95c0ac180 0x7ff95c0ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:36.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.545+0000 7ff966ffd700 1 -- 172.21.15.92:0/3962777672 <== osd.1 v2:172.21.15.144:6800/3690180636 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff95c0aec50 con 0x7ff95c0ac180 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.579+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 --> [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff970062ed0 con 0x7ff95c0ac180 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.579+0000 7ff966ffd700 1 -- 172.21.15.92:0/3962777672 <== osd.1 v2:172.21.15.144:6800/3690180636 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff970062ed0 con 0x7ff95c0ac180 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 >> [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] conn(0x7ff95c0ac180 msgr2=0x7ff95c0ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] conn(0x7ff95c0ac180 0x7ff95c0ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff95c0a29a0 msgr2=0x7ff95c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff95c0a29a0 0x7ff95c0a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff9701999e0 tx=0x7ff96c009260 comp rx=0 tx=0).stop 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9701064f0 msgr2=0x7ff9701988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 --2- 172.21.15.92:0/3962777672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9701064f0 0x7ff9701988a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff96800d010 tx=0x7ff9680079a0 comp rx=0 tx=0).stop 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff9766fc700 1 -- 172.21.15.92:0/3962777672 reap_dead start 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 shutdown_connections 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 >> 172.21.15.92:0/3962777672 conn(0x7ff970077260 msgr2=0x7ff97006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 shutdown_connections 2024-03-19T01:51:36.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:36.580+0000 7ff97815f700 1 -- 172.21.15.92:0/3962777672 wait complete. 2024-03-19T01:51:36.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:36 smithi092 ceph-mon[62574]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:51:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:36 smithi144 ceph-mon[67199]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:51:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:36 smithi159 ceph-mon[67110]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:51:37.892 INFO:teuthology.orchestra.run.smithi092.stdout:77309411397 2024-03-19T01:51:37.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-19T01:51:38.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.082+0000 7f6588c72700 1 -- 172.21.15.92:0/3183961747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6584107680 msgr2=0x7f658410f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:38.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.082+0000 7f6588c72700 1 --2- 172.21.15.92:0/3183961747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6584107680 0x7f658410f200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f65740099c0 tx=0x7f6574009cd0 comp rx=0 tx=0).stop 2024-03-19T01:51:38.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.082+0000 7f6588c72700 1 -- 172.21.15.92:0/3183961747 shutdown_connections 2024-03-19T01:51:38.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.082+0000 7f6588c72700 1 --2- 172.21.15.92:0/3183961747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6584107680 0x7f658410f200 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:38.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.082+0000 7f6588c72700 1 --2- 172.21.15.92:0/3183961747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f658406ac10 0x7f658406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:38.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.082+0000 7f6588c72700 1 --2- 172.21.15.92:0/3183961747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f658406b5c0 0x7f6584106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:38.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.082+0000 7f6588c72700 1 -- 172.21.15.92:0/3183961747 >> 172.21.15.92:0/3183961747 conn(0x7f6584078180 msgr2=0x7f6584078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:38.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.082+0000 7f6588c72700 1 -- 172.21.15.92:0/3183961747 shutdown_connections 2024-03-19T01:51:38.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.082+0000 7f6588c72700 1 -- 172.21.15.92:0/3183961747 wait complete. 2024-03-19T01:51:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6588c72700 1 Processor -- start 2024-03-19T01:51:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6588c72700 1 -- start start 2024-03-19T01:51:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6588c72700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f658406ac10 0x7f65841012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6588c72700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f658406b5c0 0x7f6584101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6588c72700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6584107680 0x7f6584101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:38.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6588c72700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6584112620 con 0x7f658406ac10 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6588c72700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65841124a0 con 0x7f6584107680 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6588c72700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65841127a0 con 0x7f658406b5c0 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f658259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f658406ac10 0x7f65841012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f658259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f658406ac10 0x7f65841012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33452/0 (socket says 172.21.15.92:33452) 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f658259c700 1 -- 172.21.15.92:0/1355775161 learned_addr learned my addr 172.21.15.92:0/1355775161 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6581d9b700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f658406b5c0 0x7f6584101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6582d9d700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6584107680 0x7f6584101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f658259c700 1 -- 172.21.15.92:0/1355775161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f658406b5c0 msgr2=0x7f6584101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f658259c700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f658406b5c0 0x7f6584101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f658259c700 1 -- 172.21.15.92:0/1355775161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6584107680 msgr2=0x7f6584101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f658259c700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6584107680 0x7f6584101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f658259c700 1 -- 172.21.15.92:0/1355775161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65841a6fd0 con 0x7f658406ac10 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.083+0000 7f6582d9d700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6584107680 0x7f6584101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.084+0000 7f658259c700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f658406ac10 0x7f65841012d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f657c00ef10 tx=0x7f657c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:38.085 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.084+0000 7f65737fe700 1 -- 172.21.15.92:0/1355775161 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c00c9c0 con 0x7f658406ac10 2024-03-19T01:51:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.084+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6584198830 con 0x7f658406ac10 2024-03-19T01:51:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.084+0000 7f65737fe700 1 -- 172.21.15.92:0/1355775161 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657c019070 con 0x7f658406ac10 2024-03-19T01:51:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.084+0000 7f65737fe700 1 -- 172.21.15.92:0/1355775161 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c007e60 con 0x7f658406ac10 2024-03-19T01:51:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.084+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6584198dc0 con 0x7f658406ac10 2024-03-19T01:51:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.085+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6584062a20 con 0x7f658406ac10 2024-03-19T01:51:38.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.086+0000 7f65737fe700 1 -- 172.21.15.92:0/1355775161 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f657c012050 con 0x7f658406ac10 2024-03-19T01:51:38.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.086+0000 7f65737fe700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f656c0a2920 0x7f656c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:38.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.087+0000 7f6581d9b700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f656c0a2920 0x7f656c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:38.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.087+0000 7f65737fe700 1 -- 172.21.15.92:0/1355775161 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f657c0ce9e0 con 0x7f658406ac10 2024-03-19T01:51:38.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.087+0000 7f65737fe700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] conn(0x7f656c0ac100 0x7f656c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:38.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.087+0000 7f65737fe700 1 -- 172.21.15.92:0/1355775161 --> [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f657c023e50 con 0x7f656c0ac100 2024-03-19T01:51:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.087+0000 7f65737fe700 1 -- 172.21.15.92:0/1355775161 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f657c0d32c0 con 0x7f658406ac10 2024-03-19T01:51:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.087+0000 7f6582d9d700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] conn(0x7f656c0ac100 0x7f656c0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.087+0000 7f6581d9b700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f656c0a2920 0x7f656c0a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6584100db0 tx=0x7f6578018040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:38.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.088+0000 7f6582d9d700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] conn(0x7f656c0ac100 0x7f656c0ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:38.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.089+0000 7f65737fe700 1 -- 172.21.15.92:0/1355775161 <== osd.10 v2:172.21.15.144:6824/3069035738 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f657c023e50 con 0x7f656c0ac100 2024-03-19T01:51:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.107+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 --> [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6584062e20 con 0x7f656c0ac100 2024-03-19T01:51:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.108+0000 7f65737fe700 1 -- 172.21.15.92:0/1355775161 <== osd.10 v2:172.21.15.144:6824/3069035738 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6584062e20 con 0x7f656c0ac100 2024-03-19T01:51:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.108+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 >> [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] conn(0x7f656c0ac100 msgr2=0x7f656c0ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.108+0000 7f6588c72700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] conn(0x7f656c0ac100 0x7f656c0ae520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.109+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f656c0a2920 msgr2=0x7f656c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.109+0000 7f6588c72700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f656c0a2920 0x7f656c0a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6584100db0 tx=0x7f6578018040 comp rx=0 tx=0).stop 2024-03-19T01:51:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.109+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f658406ac10 msgr2=0x7f65841012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.109+0000 7f6588c72700 1 --2- 172.21.15.92:0/1355775161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f658406ac10 0x7f65841012d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f657c00ef10 tx=0x7f657c00c3a0 comp rx=0 tx=0).stop 2024-03-19T01:51:38.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.109+0000 7f6582d9d700 1 -- 172.21.15.92:0/1355775161 reap_dead start 2024-03-19T01:51:38.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.109+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 shutdown_connections 2024-03-19T01:51:38.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.109+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 >> 172.21.15.92:0/1355775161 conn(0x7f6584078180 msgr2=0x7f65840ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:38.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.109+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 shutdown_connections 2024-03-19T01:51:38.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:38.109+0000 7f6588c72700 1 -- 172.21.15.92:0/1355775161 wait complete. 2024-03-19T01:51:38.779 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:38 smithi092 ceph-mon[62574]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-19T01:51:38.804 INFO:teuthology.orchestra.run.smithi092.stdout:309237645340 2024-03-19T01:51:38.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-19T01:51:38.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:38 smithi144 ceph-mon[67199]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-19T01:51:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:38 smithi159 ceph-mon[67110]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-19T01:51:39.529 INFO:teuthology.orchestra.run.smithi092.stdout:197568495665 2024-03-19T01:51:39.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-19T01:51:39.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-19T01:51:39.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-19T01:51:39.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-03-19T01:51:39.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-03-19T01:51:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-03-19T01:51:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-19T01:51:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-19T01:51:39.811 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:39 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-19T01:51:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-19T01:51:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-19T01:51:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-03-19T01:51:39.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-03-19T01:51:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-03-19T01:51:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-19T01:51:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-19T01:51:39.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:39 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-19T01:51:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-19T01:51:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-19T01:51:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-03-19T01:51:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-03-19T01:51:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-03-19T01:51:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-19T01:51:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-19T01:51:39.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:39 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-19T01:51:40.029 INFO:teuthology.orchestra.run.smithi092.stdout:55834574922 2024-03-19T01:51:40.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-19T01:51:40.066 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:40.076 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:40.076 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:40.076 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:40.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:51:40.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-19T01:51:40.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-19T01:51:40.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-03-19T01:51:40.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-03-19T01:51:40.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-03-19T01:51:40.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-19T01:51:40.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-19T01:51:40.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-19T01:51:40.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:40 smithi144 ceph-mon[67199]: osdmap e86: 12 total, 12 up, 12 in 2024-03-19T01:51:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:51:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-19T01:51:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-19T01:51:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-03-19T01:51:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-03-19T01:51:40.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-03-19T01:51:40.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-19T01:51:40.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-19T01:51:40.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-19T01:51:40.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:40 smithi159 ceph-mon[67110]: osdmap e86: 12 total, 12 up, 12 in 2024-03-19T01:51:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:51:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-19T01:51:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-19T01:51:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-03-19T01:51:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-03-19T01:51:41.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-03-19T01:51:41.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-19T01:51:41.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-19T01:51:41.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-19T01:51:41.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:40 smithi092 ceph-mon[62574]: osdmap e86: 12 total, 12 up, 12 in 2024-03-19T01:51:41.572 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:41.572 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:41.573 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:41.573 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:41.599 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:41.604 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:41.854 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:41 smithi092 ceph-mon[62574]: osdmap e87: 12 total, 12 up, 12 in 2024-03-19T01:51:41.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:41 smithi159 ceph-mon[67110]: osdmap e87: 12 total, 12 up, 12 in 2024-03-19T01:51:42.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:41 smithi144 ceph-mon[67199]: osdmap e87: 12 total, 12 up, 12 in 2024-03-19T01:51:42.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:42 smithi092 ceph-mon[62574]: pgmap v318: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-19T01:51:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:42 smithi159 ceph-mon[67110]: pgmap v318: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-19T01:51:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:42 smithi144 ceph-mon[67199]: pgmap v318: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-19T01:51:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:44 smithi159 ceph-mon[67110]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-03-19T01:51:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:44 smithi092 ceph-mon[62574]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-03-19T01:51:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:44 smithi144 ceph-mon[67199]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-03-19T01:51:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:46 smithi092 ceph-mon[62574]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-19T01:51:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:46 smithi159 ceph-mon[67110]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-19T01:51:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:46 smithi144 ceph-mon[67199]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-19T01:51:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.531+0000 7f931fe4c700 1 -- 172.21.15.92:0/2719923898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 msgr2=0x7f931806ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.531+0000 7f931fe4c700 1 --2- 172.21.15.92:0/2719923898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 0x7f931806ae90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f93140099c0 tx=0x7f9314009cd0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.531+0000 7f931fe4c700 1 -- 172.21.15.92:0/2719923898 shutdown_connections 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.531+0000 7f931fe4c700 1 --2- 172.21.15.92:0/2719923898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9318107690 0x7f931810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.531+0000 7f931fe4c700 1 --2- 172.21.15.92:0/2719923898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 0x7f931806ae90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.531+0000 7f931fe4c700 1 --2- 172.21.15.92:0/2719923898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931806b460 0x7f9318106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.531+0000 7f931fe4c700 1 -- 172.21.15.92:0/2719923898 >> 172.21.15.92:0/2719923898 conn(0x7f9318077260 msgr2=0x7f9318077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.531+0000 7f931fe4c700 1 -- 172.21.15.92:0/2719923898 shutdown_connections 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.531+0000 7f931fe4c700 1 -- 172.21.15.92:0/2719923898 wait complete. 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931fe4c700 1 Processor -- start 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931fe4c700 1 -- start start 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931fe4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 0x7f931819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931fe4c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931806b460 0x7f931819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931fe4c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9318107690 0x7f93181966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931fe4c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9318110470 con 0x7f931806aab0 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931fe4c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93181102f0 con 0x7f9318107690 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931fe4c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93181105f0 con 0x7f931806b460 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931dbe8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 0x7f931819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931dbe8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 0x7f931819c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42630/0 (socket says 172.21.15.92:42630) 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931dbe8700 1 -- 172.21.15.92:0/4209456343 learned_addr learned my addr 172.21.15.92:0/4209456343 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931d3e7700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931806b460 0x7f931819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931e3e9700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9318107690 0x7f93181966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931dbe8700 1 -- 172.21.15.92:0/4209456343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931806b460 msgr2=0x7f931819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931dbe8700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931806b460 0x7f931819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931dbe8700 1 -- 172.21.15.92:0/4209456343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9318107690 msgr2=0x7f93181966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931dbe8700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9318107690 0x7f93181966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931dbe8700 1 -- 172.21.15.92:0/4209456343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9318196fd0 con 0x7f931806aab0 2024-03-19T01:51:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931d3e7700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931806b460 0x7f931819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.532+0000 7f931e3e9700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9318107690 0x7f93181966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.533+0000 7f931dbe8700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 0x7f931819c620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9314006010 tx=0x7f931400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.533+0000 7f930effd700 1 -- 172.21.15.92:0/4209456343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314003680 con 0x7f931806aab0 2024-03-19T01:51:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.533+0000 7f930effd700 1 -- 172.21.15.92:0/4209456343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93140050d0 con 0x7f931806aab0 2024-03-19T01:51:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.533+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9318197260 con 0x7f931806aab0 2024-03-19T01:51:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.533+0000 7f930effd700 1 -- 172.21.15.92:0/4209456343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314003db0 con 0x7f931806aab0 2024-03-19T01:51:47.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.533+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93181a33d0 con 0x7f931806aab0 2024-03-19T01:51:47.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.534+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931810a840 con 0x7f931806aab0 2024-03-19T01:51:47.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.535+0000 7f930effd700 1 -- 172.21.15.92:0/4209456343 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9314005280 con 0x7f931806aab0 2024-03-19T01:51:47.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.536+0000 7f930effd700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f93040a2ac0 0x7f93040a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:47.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.536+0000 7f930effd700 1 -- 172.21.15.92:0/4209456343 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f93140cfaa0 con 0x7f931806aab0 2024-03-19T01:51:47.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.536+0000 7f931d3e7700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f93040a2ac0 0x7f93040a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:47.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.536+0000 7f931d3e7700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f93040a2ac0 0x7f93040a4f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9318197eb0 tx=0x7f9308009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:47.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.542+0000 7f930effd700 1 -- 172.21.15.92:0/4209456343 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f931405d740 con 0x7f931806aab0 2024-03-19T01:51:47.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.761+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f9318197c20 con 0x7f931806aab0 2024-03-19T01:51:47.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.761+0000 7f930effd700 1 -- 172.21.15.92:0/4209456343 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f9314013070 con 0x7f931806aab0 2024-03-19T01:51:47.761 INFO:teuthology.orchestra.run.smithi092.stdout:343597383705 2024-03-19T01:51:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f93040a2ac0 msgr2=0x7f93040a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f93040a2ac0 0x7f93040a4f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9318197eb0 tx=0x7f9308009260 comp rx=0 tx=0).stop 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 msgr2=0x7f931819c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 0x7f931819c620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9314006010 tx=0x7f931400f690 comp rx=0 tx=0).stop 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 shutdown_connections 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f93040a2ac0 0x7f93040a4f80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9318107690 0x7f93181966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f931806aab0 0x7f931819c620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 --2- 172.21.15.92:0/4209456343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931806b460 0x7f931819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 >> 172.21.15.92:0/4209456343 conn(0x7f9318077260 msgr2=0x7f93180ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 shutdown_connections 2024-03-19T01:51:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:47.764+0000 7f931fe4c700 1 -- 172.21.15.92:0/4209456343 wait complete. 2024-03-19T01:51:48.513 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:48 smithi159 ceph-mon[67110]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 95 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:51:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:48 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4209456343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-19T01:51:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:48 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:51:49.006 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:48 smithi092 ceph-mon[62574]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 95 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:51:49.006 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:48 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4209456343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-19T01:51:49.007 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:48 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:51:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:48 smithi144 ceph-mon[67199]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 95 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:51:49.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:48 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4209456343' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-19T01:51:49.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:48 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:51:49.297 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383702 got 343597383705 for osd.11 2024-03-19T01:51:49.297 DEBUG:teuthology.parallel:result is None 2024-03-19T01:51:49.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:49 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4141351161' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-19T01:51:49.876 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:49 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4141351161' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-19T01:51:49.943 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:51:49.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:49 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4141351161' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-19T01:51:50.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:50 smithi092 ceph-mon[62574]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 77 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:51:50.769 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:50.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:50.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:50.770 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:50 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:50 smithi159 ceph-mon[67110]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 77 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:51:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:50 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:50 smithi144 ceph-mon[67199]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 77 B/s, 1 keys/s, 1 objects/s recovering 2024-03-19T01:51:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:51.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:50 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.529+0000 7fc13b0e6700 1 -- 172.21.15.92:0/2406899638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc13410e0a0 msgr2=0x7fc134115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.529+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/2406899638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc13410e0a0 0x7fc134115cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00caa0 tx=0x7fc12c009f90 comp rx=0 tx=0).stop 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.530+0000 7fc13b0e6700 1 -- 172.21.15.92:0/2406899638 shutdown_connections 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.530+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/2406899638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc13410e0a0 0x7fc134115cb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.530+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/2406899638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc134109570 0x7fc134109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.530+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/2406899638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc134109f20 0x7fc13410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.530+0000 7fc13b0e6700 1 -- 172.21.15.92:0/2406899638 >> 172.21.15.92:0/2406899638 conn(0x7fc13406d450 msgr2=0x7fc13406d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.530+0000 7fc13b0e6700 1 -- 172.21.15.92:0/2406899638 shutdown_connections 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.530+0000 7fc13b0e6700 1 -- 172.21.15.92:0/2406899638 wait complete. 2024-03-19T01:51:51.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.531+0000 7fc13b0e6700 1 Processor -- start 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.531+0000 7fc13b0e6700 1 -- start start 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc13b0e6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc134109570 0x7fc13419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc13b0e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc134109f20 0x7fc13419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc13b0e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc13410e0a0 0x7fc1341a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc138e82700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc134109570 0x7fc13419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc138e82700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc134109570 0x7fc13419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:38790/0 (socket says 172.21.15.92:38790) 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc138e82700 1 -- 172.21.15.92:0/4141351161 learned_addr learned my addr 172.21.15.92:0/4141351161 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc134118ca0 con 0x7fc13410e0a0 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc139683700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc13410e0a0 0x7fc1341a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc133fff700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc134109f20 0x7fc13419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc134118b20 con 0x7fc134109f20 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.532+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc134118e20 con 0x7fc134109570 2024-03-19T01:51:51.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc138e82700 1 -- 172.21.15.92:0/4141351161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc134109f20 msgr2=0x7fc13419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc138e82700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc134109f20 0x7fc13419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc138e82700 1 -- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc13410e0a0 msgr2=0x7fc1341a3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc138e82700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc13410e0a0 0x7fc1341a3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc138e82700 1 -- 172.21.15.92:0/4141351161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1341a3ec0 con 0x7fc134109570 2024-03-19T01:51:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc133fff700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc134109f20 0x7fc13419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:51.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc138e82700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc134109570 0x7fc13419eeb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c9b0 tx=0x7fc12800ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:51.443 INFO:teuthology.orchestra.run.smithi092.stdout:141733920829 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc131ffb700 1 -- 172.21.15.92:0/4141351161 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12801e070 con 0x7fc134109570 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc131ffb700 1 -- 172.21.15.92:0/4141351161 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc12800ce50 con 0x7fc134109570 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1341a41b0 con 0x7fc134109570 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc131ffb700 1 -- 172.21.15.92:0/4141351161 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128019790 con 0x7fc134109570 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.533+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1341aba30 con 0x7fc134109570 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.535+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc134068fd0 con 0x7fc134109570 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.536+0000 7fc131ffb700 1 -- 172.21.15.92:0/4141351161 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc128021070 con 0x7fc134109570 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.537+0000 7fc131ffb700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc1240a29a0 0x7fc1240a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.537+0000 7fc131ffb700 1 -- 172.21.15.92:0/4141351161 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc1280d2530 con 0x7fc134109570 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.537+0000 7fc133fff700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc1240a29a0 0x7fc1240a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:51.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.538+0000 7fc133fff700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc1240a29a0 0x7fc1240a4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc13406dec0 tx=0x7fc120009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.542+0000 7fc131ffb700 1 -- 172.21.15.92:0/4141351161 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc12809f590 con 0x7fc134109570 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.766+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc1341abc00 con 0x7fc134109570 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.767+0000 7fc131ffb700 1 -- 172.21.15.92:0/4141351161 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc12809ee90 con 0x7fc134109570 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc1240a29a0 msgr2=0x7fc1240a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc1240a29a0 0x7fc1240a4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc13406dec0 tx=0x7fc120009260 comp rx=0 tx=0).stop 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc134109570 msgr2=0x7fc13419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc134109570 0x7fc13419eeb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c9b0 tx=0x7fc12800ccc0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 shutdown_connections 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc1240a29a0 0x7fc1240a4e60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc13410e0a0 0x7fc1341a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc134109570 0x7fc13419eeb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 --2- 172.21.15.92:0/4141351161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc134109f20 0x7fc13419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 >> 172.21.15.92:0/4141351161 conn(0x7fc13406d450 msgr2=0x7fc134071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 shutdown_connections 2024-03-19T01:51:51.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:48.769+0000 7fc13b0e6700 1 -- 172.21.15.92:0/4141351161 wait complete. 2024-03-19T01:51:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:52 smithi092 ceph-mon[62574]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 70 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:52.898 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:52 smithi144 ceph-mon[67199]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 70 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:52 smithi159 ceph-mon[67110]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 70 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:54.134 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920824 got 141733920829 for osd.4 2024-03-19T01:51:54.134 DEBUG:teuthology.parallel:result is None 2024-03-19T01:51:54.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:54 smithi092 ceph-mon[62574]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 63 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:54 smithi159 ceph-mon[67110]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 63 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:55.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:54 smithi144 ceph-mon[67199]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 63 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:55.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.746+0000 7f52260e6700 1 -- 172.21.15.92:0/100512767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220108340 msgr2=0x7f5220108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:55.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.746+0000 7f52260e6700 1 --2- 172.21.15.92:0/100512767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220108340 0x7f5220108720 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f52100099c0 tx=0x7f5210009cd0 comp rx=0 tx=0).stop 2024-03-19T01:51:55.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.746+0000 7f52260e6700 1 -- 172.21.15.92:0/100512767 shutdown_connections 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.746+0000 7f52260e6700 1 --2- 172.21.15.92:0/100512767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5220102d20 0x7f522010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.746+0000 7f52260e6700 1 --2- 172.21.15.92:0/100512767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220108340 0x7f5220108720 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.746+0000 7f52260e6700 1 --2- 172.21.15.92:0/100512767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5220102360 0x7f52201027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.746+0000 7f52260e6700 1 -- 172.21.15.92:0/100512767 >> 172.21.15.92:0/100512767 conn(0x7f5220079440 msgr2=0x7f5220079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.747+0000 7f52260e6700 1 -- 172.21.15.92:0/100512767 shutdown_connections 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.747+0000 7f52260e6700 1 -- 172.21.15.92:0/100512767 wait complete. 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.747+0000 7f52260e6700 1 Processor -- start 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.747+0000 7f52260e6700 1 -- start start 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f52260e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220102360 0x7f522019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f52260e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5220102d20 0x7f522019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f52260e6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5220108340 0x7f52201987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f52260e6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f52201125f0 con 0x7f5220102360 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f52260e6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5220112470 con 0x7f5220102d20 2024-03-19T01:51:55.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f52260e6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5220112770 con 0x7f5220108340 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220102360 0x7f522019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220102360 0x7f522019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45286/0 (socket says 172.21.15.92:45286) 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521f7fe700 1 -- 172.21.15.92:0/391998788 learned_addr learned my addr 172.21.15.92:0/391998788 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f5217fff700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5220102d20 0x7f522019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521ffff700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5220108340 0x7f52201987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521f7fe700 1 -- 172.21.15.92:0/391998788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5220108340 msgr2=0x7f52201987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521f7fe700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5220108340 0x7f52201987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521f7fe700 1 -- 172.21.15.92:0/391998788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5220102d20 msgr2=0x7f522019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521f7fe700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5220102d20 0x7f522019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521f7fe700 1 -- 172.21.15.92:0/391998788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5220198fd0 con 0x7f5220102360 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521f7fe700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220102360 0x7f522019e670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52100059d0 tx=0x7f521000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521d7fa700 1 -- 172.21.15.92:0/391998788 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210027070 con 0x7f5220102360 2024-03-19T01:51:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521d7fa700 1 -- 172.21.15.92:0/391998788 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5210003680 con 0x7f5220102360 2024-03-19T01:51:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5220199260 con 0x7f5220102360 2024-03-19T01:51:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f521d7fa700 1 -- 172.21.15.92:0/391998788 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52100173f0 con 0x7f5220102360 2024-03-19T01:51:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.748+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52201a54d0 con 0x7f5220102360 2024-03-19T01:51:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.749+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52201039f0 con 0x7f5220102360 2024-03-19T01:51:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.750+0000 7f521d7fa700 1 -- 172.21.15.92:0/391998788 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f521002a070 con 0x7f5220102360 2024-03-19T01:51:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.751+0000 7f521d7fa700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f520c0a2920 0x7f520c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.751+0000 7f5217fff700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f520c0a2920 0x7f520c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.751+0000 7f521d7fa700 1 -- 172.21.15.92:0/391998788 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f52100cf740 con 0x7f5220102360 2024-03-19T01:51:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.752+0000 7f5217fff700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f520c0a2920 0x7f520c0a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f52200799d0 tx=0x7f5208009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:55.756+0000 7f521d7fa700 1 -- 172.21.15.92:0/391998788 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5210013070 con 0x7f5220102360 2024-03-19T01:51:56.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.000+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f5220199d20 con 0x7f5220102360 2024-03-19T01:51:56.000 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.000+0000 7f521d7fa700 1 -- 172.21.15.92:0/391998788 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f521009c720 con 0x7f5220102360 2024-03-19T01:51:56.001 INFO:teuthology.orchestra.run.smithi092.stdout:287762808868 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f520c0a2920 msgr2=0x7f520c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f520c0a2920 0x7f520c0a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f52200799d0 tx=0x7f5208009260 comp rx=0 tx=0).stop 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220102360 msgr2=0x7f522019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220102360 0x7f522019e670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52100059d0 tx=0x7f521000fa00 comp rx=0 tx=0).stop 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 shutdown_connections 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f520c0a2920 0x7f520c0a4de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5220108340 0x7f52201987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5220102360 0x7f522019e670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 --2- 172.21.15.92:0/391998788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5220102d20 0x7f522019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.002+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 >> 172.21.15.92:0/391998788 conn(0x7f5220079440 msgr2=0x7f5220100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.003+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 shutdown_connections 2024-03-19T01:51:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.003+0000 7f52260e6700 1 -- 172.21.15.92:0/391998788 wait complete. 2024-03-19T01:51:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.269+0000 7ff52ec4b700 1 -- 172.21.15.92:0/1035582447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5281064f0 msgr2=0x7ff52810e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.269+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/1035582447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5281064f0 0x7ff52810e0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff520009fc0 tx=0x7ff52000b330 comp rx=0 tx=0).stop 2024-03-19T01:51:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.269+0000 7ff52ec4b700 1 -- 172.21.15.92:0/1035582447 shutdown_connections 2024-03-19T01:51:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.269+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/1035582447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5281064f0 0x7ff52810e0c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.269+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/1035582447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5281014a0 0x7ff528101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.269+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/1035582447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff528101e50 0x7ff528105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.269+0000 7ff52ec4b700 1 -- 172.21.15.92:0/1035582447 >> 172.21.15.92:0/1035582447 conn(0x7ff528077260 msgr2=0x7ff528077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.269+0000 7ff52ec4b700 1 -- 172.21.15.92:0/1035582447 shutdown_connections 2024-03-19T01:51:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.269+0000 7ff52ec4b700 1 -- 172.21.15.92:0/1035582447 wait complete. 2024-03-19T01:51:56.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52ec4b700 1 Processor -- start 2024-03-19T01:51:56.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52ec4b700 1 -- start start 2024-03-19T01:51:56.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52ec4b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5281014a0 0x7ff52819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:56.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52ec4b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff528101e50 0x7ff52819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:56.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52ec4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5281064f0 0x7ff528198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:56.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52ec4b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5281114e0 con 0x7ff5281014a0 2024-03-19T01:51:56.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52ec4b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff528111360 con 0x7ff5281064f0 2024-03-19T01:51:56.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52ec4b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff528111660 con 0x7ff528101e50 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52c9e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5281014a0 0x7ff52819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52c9e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5281014a0 0x7ff52819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45308/0 (socket says 172.21.15.92:45308) 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52c9e7700 1 -- 172.21.15.92:0/2032553363 learned_addr learned my addr 172.21.15.92:0/2032553363 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff52d1e8700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5281064f0 0x7ff528198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.270+0000 7ff527fff700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff528101e50 0x7ff52819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff52d1e8700 1 -- 172.21.15.92:0/2032553363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff528101e50 msgr2=0x7ff52819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff52d1e8700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff528101e50 0x7ff52819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff52d1e8700 1 -- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5281014a0 msgr2=0x7ff52819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff52d1e8700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5281014a0 0x7ff52819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff52d1e8700 1 -- 172.21.15.92:0/2032553363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff528198ff0 con 0x7ff5281064f0 2024-03-19T01:51:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff52c9e7700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5281014a0 0x7ff52819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff527fff700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff528101e50 0x7ff52819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff52d1e8700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5281064f0 0x7ff528198890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff52000b6c0 tx=0x7ff520004f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff525ffb700 1 -- 172.21.15.92:0/2032553363 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520021070 con 0x7ff5281064f0 2024-03-19T01:51:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff525ffb700 1 -- 172.21.15.92:0/2032553363 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff520007ac0 con 0x7ff5281064f0 2024-03-19T01:51:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff528199280 con 0x7ff5281064f0 2024-03-19T01:51:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.271+0000 7ff525ffb700 1 -- 172.21.15.92:0/2032553363 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520005040 con 0x7ff5281064f0 2024-03-19T01:51:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.272+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5281a55a0 con 0x7ff5281064f0 2024-03-19T01:51:56.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.273+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5281028f0 con 0x7ff5281064f0 2024-03-19T01:51:56.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.273+0000 7ff525ffb700 1 -- 172.21.15.92:0/2032553363 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff520024070 con 0x7ff5281064f0 2024-03-19T01:51:56.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.274+0000 7ff525ffb700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5180a2920 0x7ff5180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:56.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.274+0000 7ff52c9e7700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5180a2920 0x7ff5180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:56.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.274+0000 7ff525ffb700 1 -- 172.21.15.92:0/2032553363 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff5200cf150 con 0x7ff5281064f0 2024-03-19T01:51:56.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.274+0000 7ff52c9e7700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5180a2920 0x7ff5180a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff5281999d0 tx=0x7ff51c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:56.278 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.278+0000 7ff525ffb700 1 -- 172.21.15.92:0/2032553363 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff52009c130 con 0x7ff5281064f0 2024-03-19T01:51:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.496+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff528068fd0 con 0x7ff5281064f0 2024-03-19T01:51:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.496+0000 7ff525ffb700 1 -- 172.21.15.92:0/2032553363 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff52009ba30 con 0x7ff5281064f0 2024-03-19T01:51:56.497 INFO:teuthology.orchestra.run.smithi092.stdout:34359738451 2024-03-19T01:51:56.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5180a2920 msgr2=0x7ff5180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5180a2920 0x7ff5180a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff5281999d0 tx=0x7ff51c009260 comp rx=0 tx=0).stop 2024-03-19T01:51:56.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5281064f0 msgr2=0x7ff528198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5281064f0 0x7ff528198890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff52000b6c0 tx=0x7ff520004f40 comp rx=0 tx=0).stop 2024-03-19T01:51:56.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 shutdown_connections 2024-03-19T01:51:56.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5180a2920 0x7ff5180a4de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5281064f0 0x7ff528198890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5281014a0 0x7ff52819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 --2- 172.21.15.92:0/2032553363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff528101e50 0x7ff52819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 >> 172.21.15.92:0/2032553363 conn(0x7ff528077260 msgr2=0x7ff52806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 shutdown_connections 2024-03-19T01:51:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.499+0000 7ff52ec4b700 1 -- 172.21.15.92:0/2032553363 wait complete. 2024-03-19T01:51:56.762 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:56 smithi092 ceph-mon[62574]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 63 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:56.762 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:56 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/391998788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-19T01:51:56.762 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:56 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/11199756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-19T01:51:56.763 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:56 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2032553363' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-19T01:51:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.764+0000 7f823759e700 1 -- 172.21.15.92:0/1051257460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 msgr2=0x7f8238106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.764+0000 7f823759e700 1 --2- 172.21.15.92:0/1051257460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 0x7f8238106e40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f822c0099c0 tx=0x7f822c009cd0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.764+0000 7f823759e700 1 -- 172.21.15.92:0/1051257460 shutdown_connections 2024-03-19T01:51:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.764+0000 7f823759e700 1 --2- 172.21.15.92:0/1051257460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8238107520 0x7f823810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.764+0000 7f823759e700 1 --2- 172.21.15.92:0/1051257460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238102440 0x7f8238102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.764+0000 7f823759e700 1 --2- 172.21.15.92:0/1051257460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 0x7f8238106e40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.764+0000 7f823759e700 1 -- 172.21.15.92:0/1051257460 >> 172.21.15.92:0/1051257460 conn(0x7f82380fdd30 msgr2=0x7f8238100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.764+0000 7f823759e700 1 -- 172.21.15.92:0/1051257460 shutdown_connections 2024-03-19T01:51:56.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.764+0000 7f823759e700 1 -- 172.21.15.92:0/1051257460 wait complete. 2024-03-19T01:51:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.765+0000 7f823759e700 1 Processor -- start 2024-03-19T01:51:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.765+0000 7f823759e700 1 -- start start 2024-03-19T01:51:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823759e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8238102440 0x7f823819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 0x7f823819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823759e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238107520 0x7f82381987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f82381124e0 con 0x7f8238102df0 2024-03-19T01:51:56.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823759e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8238112360 con 0x7f8238102440 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823759e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8238112660 con 0x7f8238107520 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f8235d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 0x7f823819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f8235d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 0x7f823819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45328/0 (socket says 172.21.15.92:45328) 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8238102440 0x7f823819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f8235d9b700 1 -- 172.21.15.92:0/4188372802 learned_addr learned my addr 172.21.15.92:0/4188372802 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f8236d9d700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238107520 0x7f82381987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823659c700 1 -- 172.21.15.92:0/4188372802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238107520 msgr2=0x7f82381987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823659c700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238107520 0x7f82381987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823659c700 1 -- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 msgr2=0x7f823819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823659c700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 0x7f823819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823659c700 1 -- 172.21.15.92:0/4188372802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8238198fc0 con 0x7f8238102440 2024-03-19T01:51:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f8235d9b700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 0x7f823819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823659c700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8238102440 0x7f823819e610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f822800e9f0 tx=0x7f822800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f82277fe700 1 -- 172.21.15.92:0/4188372802 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822800c9c0 con 0x7f8238102440 2024-03-19T01:51:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f82277fe700 1 -- 172.21.15.92:0/4188372802 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822800cb60 con 0x7f8238102440 2024-03-19T01:51:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.766+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82381992b0 con 0x7f8238102440 2024-03-19T01:51:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.767+0000 7f82277fe700 1 -- 172.21.15.92:0/4188372802 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228018680 con 0x7f8238102440 2024-03-19T01:51:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.767+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381a5470 con 0x7f8238102440 2024-03-19T01:51:56.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.768+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8238103920 con 0x7f8238102440 2024-03-19T01:51:56.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.769+0000 7f82277fe700 1 -- 172.21.15.92:0/4188372802 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f8228010430 con 0x7f8238102440 2024-03-19T01:51:56.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.769+0000 7f82277fe700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f82200a29a0 0x7f82200a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:56.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.769+0000 7f8235d9b700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f82200a29a0 0x7f82200a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:56.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.770+0000 7f82277fe700 1 -- 172.21.15.92:0/4188372802 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f82280cec70 con 0x7f8238102440 2024-03-19T01:51:56.770 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.770+0000 7f8235d9b700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f82200a29a0 0x7f82200a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8238101430 tx=0x7f822c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:56.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.774+0000 7f82277fe700 1 -- 172.21.15.92:0/4188372802 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f822809bbc0 con 0x7f8238102440 2024-03-19T01:51:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:56 smithi159 ceph-mon[67110]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 63 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:56 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/391998788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-19T01:51:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:56 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/11199756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-19T01:51:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:56 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2032553363' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-19T01:51:56.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.978+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f823802dbb0 con 0x7f8238102440 2024-03-19T01:51:56.979 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.978+0000 7f82277fe700 1 -- 172.21.15.92:0/4188372802 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f822809b4c0 con 0x7f8238102440 2024-03-19T01:51:56.979 INFO:teuthology.orchestra.run.smithi092.stdout:309237645344 2024-03-19T01:51:56.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f82200a29a0 msgr2=0x7f82200a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f82200a29a0 0x7f82200a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8238101430 tx=0x7f822c005fb0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8238102440 msgr2=0x7f823819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:56.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8238102440 0x7f823819e610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f822800e9f0 tx=0x7f822800ed00 comp rx=0 tx=0).stop 2024-03-19T01:51:56.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 shutdown_connections 2024-03-19T01:51:56.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f82200a29a0 0x7f82200a4e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238107520 0x7f82381987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8238102440 0x7f823819e610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 --2- 172.21.15.92:0/4188372802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8238102df0 0x7f823819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:56.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.981+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 >> 172.21.15.92:0/4188372802 conn(0x7f82380fdd30 msgr2=0x7f82381000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:56.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.982+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 shutdown_connections 2024-03-19T01:51:56.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.982+0000 7f823759e700 1 -- 172.21.15.92:0/4188372802 wait complete. 2024-03-19T01:51:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.008+0000 7ff588e06700 1 -- 172.21.15.92:0/1351697145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff58410edf0 msgr2=0x7ff5841111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.008+0000 7ff588e06700 1 --2- 172.21.15.92:0/1351697145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff58410edf0 0x7ff5841111e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00b0a0 tx=0x7ff57c00b3b0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.008+0000 7ff588e06700 1 -- 172.21.15.92:0/1351697145 shutdown_connections 2024-03-19T01:51:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.008+0000 7ff588e06700 1 --2- 172.21.15.92:0/1351697145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff58410edf0 0x7ff5841111e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.008+0000 7ff588e06700 1 --2- 172.21.15.92:0/1351697145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff58406a850 0x7ff58406ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.008+0000 7ff588e06700 1 --2- 172.21.15.92:0/1351697145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff58406b170 0x7ff58410e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.008+0000 7ff588e06700 1 -- 172.21.15.92:0/1351697145 >> 172.21.15.92:0/1351697145 conn(0x7ff584077260 msgr2=0x7ff584077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.008+0000 7ff588e06700 1 -- 172.21.15.92:0/1351697145 shutdown_connections 2024-03-19T01:51:57.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.008+0000 7ff588e06700 1 -- 172.21.15.92:0/1351697145 wait complete. 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff588e06700 1 Processor -- start 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff588e06700 1 -- start start 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff588e06700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff58406a850 0x7ff5841a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff588e06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff58406b170 0x7ff5841a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff588e06700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff58410edf0 0x7ff58419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff581d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff58406b170 0x7ff5841a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff588e06700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5841147e0 con 0x7ff58406b170 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff588e06700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff584114660 con 0x7ff58406a850 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff588e06700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff584114960 con 0x7ff58410edf0 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff58259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff58406a850 0x7ff5841a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff58259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff58406a850 0x7ff5841a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:38314/0 (socket says 172.21.15.92:38314) 2024-03-19T01:51:57.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.009+0000 7ff58259c700 1 -- 172.21.15.92:0/481465008 learned_addr learned my addr 172.21.15.92:0/481465008 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff582d9d700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff58410edf0 0x7ff58419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff58259c700 1 -- 172.21.15.92:0/481465008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff58410edf0 msgr2=0x7ff58419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff58259c700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff58410edf0 0x7ff58419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff58259c700 1 -- 172.21.15.92:0/481465008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff58406b170 msgr2=0x7ff5841a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff58259c700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff58406b170 0x7ff5841a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff58259c700 1 -- 172.21.15.92:0/481465008 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58419b240 con 0x7ff58406a850 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff58259c700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff58406a850 0x7ff5841a08d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff57400e9a0 tx=0x7ff57400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff5737fe700 1 -- 172.21.15.92:0/481465008 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57400cbd0 con 0x7ff58406a850 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff5737fe700 1 -- 172.21.15.92:0/481465008 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57400cd70 con 0x7ff58406a850 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58419b530 con 0x7ff58406a850 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff5737fe700 1 -- 172.21.15.92:0/481465008 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574018750 con 0x7ff58406a850 2024-03-19T01:51:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.010+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5841a77a0 con 0x7ff58406a850 2024-03-19T01:51:57.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.011+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff584068fd0 con 0x7ff58406a850 2024-03-19T01:51:57.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.012+0000 7ff5737fe700 1 -- 172.21.15.92:0/481465008 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff574010430 con 0x7ff58406a850 2024-03-19T01:51:57.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.013+0000 7ff5737fe700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff56c0a29a0 0x7ff56c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:57.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.013+0000 7ff581d9b700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff56c0a29a0 0x7ff56c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:57.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.013+0000 7ff5737fe700 1 -- 172.21.15.92:0/481465008 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff5740cee20 con 0x7ff58406a850 2024-03-19T01:51:57.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.014+0000 7ff581d9b700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff56c0a29a0 0x7ff56c0a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5841026c0 tx=0x7ff578009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:57.018 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.018+0000 7ff5737fe700 1 -- 172.21.15.92:0/481465008 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff57409bd70 con 0x7ff58406a850 2024-03-19T01:51:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:56 smithi144 ceph-mon[67199]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 63 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:56 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/391998788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-19T01:51:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:56 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/11199756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-19T01:51:57.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:56 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2032553363' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.167+0000 7f2dd7d59700 1 -- 172.21.15.92:0/1290470122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd0107690 msgr2=0x7f2dd010f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.167+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/1290470122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd0107690 0x7f2dd010f210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009920 tx=0x7f2dc0009c30 comp rx=0 tx=0).stop 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.167+0000 7f2dd7d59700 1 -- 172.21.15.92:0/1290470122 shutdown_connections 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.167+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/1290470122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd0107690 0x7f2dd010f210 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.167+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/1290470122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dd006ac70 0x7f2dd006b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.167+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/1290470122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dd006b620 0x7f2dd0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.167+0000 7f2dd7d59700 1 -- 172.21.15.92:0/1290470122 >> 172.21.15.92:0/1290470122 conn(0x7f2dd0077230 msgr2=0x7f2dd0077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.167+0000 7f2dd7d59700 1 -- 172.21.15.92:0/1290470122 shutdown_connections 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.167+0000 7f2dd7d59700 1 -- 172.21.15.92:0/1290470122 wait complete. 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd7d59700 1 Processor -- start 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd7d59700 1 -- start start 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd7d59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dd006ac70 0x7f2dd0101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd7d59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dd006b620 0x7f2dd0101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:57.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd7d59700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd0107690 0x7f2dd0101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd7d59700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0112630 con 0x7f2dd006b620 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd7d59700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01124b0 con 0x7f2dd006ac70 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd7d59700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01127b0 con 0x7f2dd0107690 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd5af5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dd006ac70 0x7f2dd0101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd5af5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dd006ac70 0x7f2dd0101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:38260/0 (socket says 172.21.15.92:38260) 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd5af5700 1 -- 172.21.15.92:0/11199756 learned_addr learned my addr 172.21.15.92:0/11199756 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd52f4700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dd006b620 0x7f2dd0101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.168+0000 7f2dd62f6700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd0107690 0x7f2dd0101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dd5af5700 1 -- 172.21.15.92:0/11199756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd0107690 msgr2=0x7f2dd0101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dd5af5700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd0107690 0x7f2dd0101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dd5af5700 1 -- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dd006b620 msgr2=0x7f2dd0101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dd5af5700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dd006b620 0x7f2dd0101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dd5af5700 1 -- 172.21.15.92:0/11199756 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd01a7030 con 0x7f2dd006ac70 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stdout:107374182467 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dd62f6700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd0107690 0x7f2dd0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dd52f4700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dd006b620 0x7f2dd0101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dd5af5700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dd006ac70 0x7f2dd0101310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00e9c0 tx=0x7f2dcc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dc6ffd700 1 -- 172.21.15.92:0/11199756 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00cbd0 con 0x7f2dd006ac70 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dc6ffd700 1 -- 172.21.15.92:0/11199756 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dcc00cd70 con 0x7f2dd006ac70 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dc6ffd700 1 -- 172.21.15.92:0/11199756 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc005660 con 0x7f2dd006ac70 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.169+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd01a7320 con 0x7f2dd006ac70 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.170+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd01a78b0 con 0x7f2dd006ac70 2024-03-19T01:51:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.170+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd0103a40 con 0x7f2dd006ac70 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.172+0000 7f2dc6ffd700 1 -- 172.21.15.92:0/11199756 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f2dcc004750 con 0x7f2dd006ac70 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.172+0000 7f2dc6ffd700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2dbc0a29a0 0x7f2dbc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.172+0000 7f2dd52f4700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2dbc0a29a0 0x7f2dbc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.172+0000 7f2dc6ffd700 1 -- 172.21.15.92:0/11199756 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2dcc0ceef0 con 0x7f2dd006ac70 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.173+0000 7f2dd52f4700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2dbc0a29a0 0x7f2dbc0a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0100ed0 tx=0x7f2dc800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.176+0000 7f2dc6ffd700 1 -- 172.21.15.92:0/11199756 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2dcc053740 con 0x7f2dd006ac70 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.394+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2dd002dbb0 con 0x7f2dd006ac70 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.395+0000 7f2dc6ffd700 1 -- 172.21.15.92:0/11199756 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2dcc09be40 con 0x7f2dd006ac70 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2dbc0a29a0 msgr2=0x7f2dbc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2dbc0a29a0 0x7f2dbc0a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0100ed0 tx=0x7f2dc800b040 comp rx=0 tx=0).stop 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dd006ac70 msgr2=0x7f2dd0101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dd006ac70 0x7f2dd0101310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00e9c0 tx=0x7f2dcc00ecd0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 shutdown_connections 2024-03-19T01:51:57.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2dbc0a29a0 0x7f2dbc0a4e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd0107690 0x7f2dd0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dd006ac70 0x7f2dd0101310 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 --2- 172.21.15.92:0/11199756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2dd006b620 0x7f2dd0101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 >> 172.21.15.92:0/11199756 conn(0x7f2dd0077230 msgr2=0x7f2dd00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:57.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 shutdown_connections 2024-03-19T01:51:57.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:56.397+0000 7f2dd7d59700 1 -- 172.21.15.92:0/11199756 wait complete. 2024-03-19T01:51:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.222+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff58402dbb0 con 0x7ff58406a850 2024-03-19T01:51:57.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.223+0000 7ff5737fe700 1 -- 172.21.15.92:0/481465008 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff57409b670 con 0x7ff58406a850 2024-03-19T01:51:57.263 INFO:teuthology.orchestra.run.smithi092.stdout:253403070506 2024-03-19T01:51:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff56c0a29a0 msgr2=0x7ff56c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff56c0a29a0 0x7ff56c0a4e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5841026c0 tx=0x7ff578009260 comp rx=0 tx=0).stop 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff58406a850 msgr2=0x7ff5841a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff58406a850 0x7ff5841a08d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff57400e9a0 tx=0x7ff57400ecb0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 shutdown_connections 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff56c0a29a0 0x7ff56c0a4e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff58410edf0 0x7ff58419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff58406a850 0x7ff5841a08d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 --2- 172.21.15.92:0/481465008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff58406b170 0x7ff5841a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 >> 172.21.15.92:0/481465008 conn(0x7ff584077260 msgr2=0x7ff584110240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 shutdown_connections 2024-03-19T01:51:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:57.225+0000 7ff588e06700 1 -- 172.21.15.92:0/481465008 wait complete. 2024-03-19T01:51:57.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:57 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4188372802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-19T01:51:57.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:57 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/481465008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-19T01:51:57.988 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:57 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4188372802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-19T01:51:57.988 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:57 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/481465008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-19T01:51:58.021 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070499 got 253403070506 for osd.8 2024-03-19T01:51:58.021 DEBUG:teuthology.parallel:result is None 2024-03-19T01:51:58.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:57 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4188372802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-19T01:51:58.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:57 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/481465008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-19T01:51:58.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:51:58 smithi092 ceph-mon[62574]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 63 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:51:58 smithi159 ceph-mon[67110]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 63 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:51:58 smithi144 ceph-mon[67199]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 63 B/s, 0 keys/s, 0 objects/s recovering 2024-03-19T01:51:59.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808861 got 287762808868 for osd.9 2024-03-19T01:51:59.977 DEBUG:teuthology.parallel:result is None 2024-03-19T01:52:00.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.932+0000 7f3a3576f700 1 -- 172.21.15.92:0/1032587035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301014a0 msgr2=0x7f3a30101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:00.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.932+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1032587035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301014a0 0x7f3a30101880 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3a200099c0 tx=0x7f3a20009cd0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.932+0000 7f3a3576f700 1 -- 172.21.15.92:0/1032587035 shutdown_connections 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.932+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1032587035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a301064f0 0x7f3a3010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.932+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1032587035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301014a0 0x7f3a30101880 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.932+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1032587035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a30101e50 0x7f3a30105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.932+0000 7f3a3576f700 1 -- 172.21.15.92:0/1032587035 >> 172.21.15.92:0/1032587035 conn(0x7f3a30077260 msgr2=0x7f3a30077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.932+0000 7f3a3576f700 1 -- 172.21.15.92:0/1032587035 shutdown_connections 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.932+0000 7f3a3576f700 1 -- 172.21.15.92:0/1032587035 wait complete. 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a3576f700 1 Processor -- start 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a3576f700 1 -- start start 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a3576f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a301014a0 0x7f3a30199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a3576f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a30101e50 0x7f3a30199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a3576f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301064f0 0x7f3a301a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:00.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a3576f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301114f0 con 0x7f3a301064f0 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a3576f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3a30111370 con 0x7f3a30101e50 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a3576f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a30111670 con 0x7f3a301014a0 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a2f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301064f0 0x7f3a301a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a2f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301064f0 0x7f3a301a49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45358/0 (socket says 172.21.15.92:45358) 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a2e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a30101e50 0x7f3a30199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a2f7fe700 1 -- 172.21.15.92:0/1020357861 learned_addr learned my addr 172.21.15.92:0/1020357861 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.933+0000 7f3a2effd700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a301014a0 0x7f3a30199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a2f7fe700 1 -- 172.21.15.92:0/1020357861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a301014a0 msgr2=0x7f3a30199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a2f7fe700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a301014a0 0x7f3a30199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a2f7fe700 1 -- 172.21.15.92:0/1020357861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a30101e50 msgr2=0x7f3a30199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a2f7fe700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a30101e50 0x7f3a30199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a2f7fe700 1 -- 172.21.15.92:0/1020357861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a301a4ef0 con 0x7f3a301064f0 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a2e7fc700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a30101e50 0x7f3a30199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:00.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a2f7fe700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301064f0 0x7f3a301a49b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800ef30 tx=0x7f3a2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a27fff700 1 -- 172.21.15.92:0/1020357861 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a280092e0 con 0x7f3a301064f0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a301a50c0 con 0x7f3a301064f0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a27fff700 1 -- 172.21.15.92:0/1020357861 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a28004bb0 con 0x7f3a301064f0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a27fff700 1 -- 172.21.15.92:0/1020357861 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28004280 con 0x7f3a301064f0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.934+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a301a5570 con 0x7f3a301064f0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.935+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a30068fd0 con 0x7f3a301064f0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.936+0000 7f3a27fff700 1 -- 172.21.15.92:0/1020357861 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f3a2801b030 con 0x7f3a301064f0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.937+0000 7f3a27fff700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3a1c0a2920 0x7f3a1c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.937+0000 7f3a2effd700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3a1c0a2920 0x7f3a1c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.937+0000 7f3a27fff700 1 -- 172.21.15.92:0/1020357861 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3a280cefd0 con 0x7f3a301064f0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.937+0000 7f3a2effd700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3a1c0a2920 0x7f3a1c0a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009990 tx=0x7f3a20005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:00.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:51:59.941+0000 7f3a27fff700 1 -- 172.21.15.92:0/1020357861 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3a28022020 con 0x7f3a301064f0 2024-03-19T01:52:00.173 INFO:teuthology.orchestra.run.smithi092.stdout:163208757307 2024-03-19T01:52:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.147+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3a3002dbb0 con 0x7f3a301064f0 2024-03-19T01:52:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.148+0000 7f3a27fff700 1 -- 172.21.15.92:0/1020357861 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3a28010040 con 0x7f3a301064f0 2024-03-19T01:52:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3a1c0a2920 msgr2=0x7f3a1c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3a1c0a2920 0x7f3a1c0a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009990 tx=0x7f3a20005a90 comp rx=0 tx=0).stop 2024-03-19T01:52:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301064f0 msgr2=0x7f3a301a49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:00.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301064f0 0x7f3a301a49b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800ef30 tx=0x7f3a2800c3a0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 shutdown_connections 2024-03-19T01:52:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3a1c0a2920 0x7f3a1c0a4de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a301064f0 0x7f3a301a49b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a301014a0 0x7f3a30199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 --2- 172.21.15.92:0/1020357861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a30101e50 0x7f3a30199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 >> 172.21.15.92:0/1020357861 conn(0x7f3a30077260 msgr2=0x7f3a301009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 shutdown_connections 2024-03-19T01:52:00.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:00.150+0000 7f3a3576f700 1 -- 172.21.15.92:0/1020357861 wait complete. 2024-03-19T01:52:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:00 smithi092 ceph-mon[62574]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:00.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:00 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1020357861' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-19T01:52:00.839 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757307 for osd.5 2024-03-19T01:52:00.840 DEBUG:teuthology.parallel:result is None 2024-03-19T01:52:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:00 smithi159 ceph-mon[67110]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:00 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1020357861' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-19T01:52:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:00 smithi144 ceph-mon[67199]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:00 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1020357861' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-19T01:52:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.704+0000 7f9d21f35700 1 -- 172.21.15.92:0/4245840989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d1c069fc0 msgr2=0x7f9d1c06a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.704+0000 7f9d21f35700 1 --2- 172.21.15.92:0/4245840989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d1c069fc0 0x7f9d1c06a420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0099c0 tx=0x7f9d0c009cd0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.704+0000 7f9d21f35700 1 -- 172.21.15.92:0/4245840989 shutdown_connections 2024-03-19T01:52:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.704+0000 7f9d21f35700 1 --2- 172.21.15.92:0/4245840989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c06a960 0x7f9d1c10f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.704+0000 7f9d21f35700 1 --2- 172.21.15.92:0/4245840989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d1c108360 0x7f9d1c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.704+0000 7f9d21f35700 1 --2- 172.21.15.92:0/4245840989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d1c069fc0 0x7f9d1c06a420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.704+0000 7f9d21f35700 1 -- 172.21.15.92:0/4245840989 >> 172.21.15.92:0/4245840989 conn(0x7f9d1c078140 msgr2=0x7f9d1c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.704+0000 7f9d21f35700 1 -- 172.21.15.92:0/4245840989 shutdown_connections 2024-03-19T01:52:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.704+0000 7f9d21f35700 1 -- 172.21.15.92:0/4245840989 wait complete. 2024-03-19T01:52:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d21f35700 1 Processor -- start 2024-03-19T01:52:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d21f35700 1 -- start start 2024-03-19T01:52:01.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d21f35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d1c069fc0 0x7f9d1c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:01.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d21f35700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d1c06a960 0x7f9d1c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:01.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d21f35700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108360 0x7f9d1c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:01.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d21f35700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c112610 con 0x7f9d1c069fc0 2024-03-19T01:52:01.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d21f35700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c112490 con 0x7f9d1c108360 2024-03-19T01:52:01.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d21f35700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c112790 con 0x7f9d1c06a960 2024-03-19T01:52:01.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d1affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d1c06a960 0x7f9d1c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d1b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d1c069fc0 0x7f9d1c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d1affd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d1c06a960 0x7f9d1c19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:58622/0 (socket says 172.21.15.92:58622) 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d1bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108360 0x7f9d1c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.705+0000 7f9d1affd700 1 -- 172.21.15.92:0/842969747 learned_addr learned my addr 172.21.15.92:0/842969747 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d1bfff700 1 -- 172.21.15.92:0/842969747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d1c06a960 msgr2=0x7f9d1c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d1bfff700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d1c06a960 0x7f9d1c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d1bfff700 1 -- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d1c069fc0 msgr2=0x7f9d1c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d1bfff700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d1c069fc0 0x7f9d1c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d1bfff700 1 -- 172.21.15.92:0/842969747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c199050 con 0x7f9d1c108360 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d1affd700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d1c06a960 0x7f9d1c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:01.709 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d1b7fe700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d1c069fc0 0x7f9d1c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d1bfff700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108360 0x7f9d1c1988f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400efc0 tx=0x7f9d1400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d18ff9700 1 -- 172.21.15.92:0/842969747 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d140092e0 con 0x7f9d1c108360 2024-03-19T01:52:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d18ff9700 1 -- 172.21.15.92:0/842969747 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d14004bb0 con 0x7f9d1c108360 2024-03-19T01:52:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c1992e0 con 0x7f9d1c108360 2024-03-19T01:52:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d18ff9700 1 -- 172.21.15.92:0/842969747 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14004280 con 0x7f9d1c108360 2024-03-19T01:52:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.706+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c1a5600 con 0x7f9d1c108360 2024-03-19T01:52:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.707+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c068fd0 con 0x7f9d1c108360 2024-03-19T01:52:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.708+0000 7f9d18ff9700 1 -- 172.21.15.92:0/842969747 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9d1401b030 con 0x7f9d1c108360 2024-03-19T01:52:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.708+0000 7f9d18ff9700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d040a2920 0x7f9d040a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.709+0000 7f9d1b7fe700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d040a2920 0x7f9d040a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.709+0000 7f9d18ff9700 1 -- 172.21.15.92:0/842969747 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9d140ce700 con 0x7f9d1c108360 2024-03-19T01:52:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.709+0000 7f9d1b7fe700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d040a2920 0x7f9d040a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c199a30 tx=0x7f9d10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.713+0000 7f9d18ff9700 1 -- 172.21.15.92:0/842969747 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9d14022020 con 0x7f9d1c108360 2024-03-19T01:52:01.873 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645340 got 309237645344 for osd.10 2024-03-19T01:52:01.874 DEBUG:teuthology.parallel:result is None 2024-03-19T01:52:01.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.941+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9d1c02dbb0 con 0x7f9d1c108360 2024-03-19T01:52:01.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.941+0000 7f9d18ff9700 1 -- 172.21.15.92:0/842969747 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d14010040 con 0x7f9d1c108360 2024-03-19T01:52:01.942 INFO:teuthology.orchestra.run.smithi092.stdout:77309411402 2024-03-19T01:52:01.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d040a2920 msgr2=0x7f9d040a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d040a2920 0x7f9d040a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c199a30 tx=0x7f9d10009260 comp rx=0 tx=0).stop 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108360 msgr2=0x7f9d1c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108360 0x7f9d1c1988f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400efc0 tx=0x7f9d1400c3a0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 shutdown_connections 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d040a2920 0x7f9d040a4de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d1c108360 0x7f9d1c1988f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d1c069fc0 0x7f9d1c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 --2- 172.21.15.92:0/842969747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d1c06a960 0x7f9d1c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 >> 172.21.15.92:0/842969747 conn(0x7f9d1c078140 msgr2=0x7f9d1c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 shutdown_connections 2024-03-19T01:52:01.944 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:01.943+0000 7f9d21f35700 1 -- 172.21.15.92:0/842969747 wait complete. 2024-03-19T01:52:01.990 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182467 for osd.3 2024-03-19T01:52:01.990 DEBUG:teuthology.parallel:result is None 2024-03-19T01:52:02.946 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:02 smithi092 ceph-mon[62574]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:02.946 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:02 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/842969747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-19T01:52:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:02 smithi159 ceph-mon[67110]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:02 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/842969747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-19T01:52:02.972 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411402 for osd.2 2024-03-19T01:52:02.972 DEBUG:teuthology.parallel:result is None 2024-03-19T01:52:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:02 smithi144 ceph-mon[67199]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:02 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/842969747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-19T01:52:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 -- 172.21.15.92:0/2324834057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 msgr2=0x7f6a04107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/2324834057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 0x7f6a04107620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009990 tx=0x7f69f4009ca0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 -- 172.21.15.92:0/2324834057 shutdown_connections 2024-03-19T01:52:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/2324834057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a04101bc0 0x7f6a0410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/2324834057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 0x7f6a04107620 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/2324834057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a04101200 0x7f6a04101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 -- 172.21.15.92:0/2324834057 >> 172.21.15.92:0/2324834057 conn(0x7f6a040781b0 msgr2=0x7f6a040785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 -- 172.21.15.92:0/2324834057 shutdown_connections 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 -- 172.21.15.92:0/2324834057 wait complete. 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 Processor -- start 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.393+0000 7f6a09f6a700 1 -- start start 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a09f6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a04101200 0x7f6a04102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a09f6a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a04101bc0 0x7f6a041033f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a09f6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 0x7f6a04103930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a09f6a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041114e0 con 0x7f6a04107240 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a09f6a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04111360 con 0x7f6a04101200 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a09f6a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04111660 con 0x7f6a04101bc0 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 0x7f6a04103930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 0x7f6a04103930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45398/0 (socket says 172.21.15.92:45398) 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a04101200 0x7f6a04102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a03fff700 1 -- 172.21.15.92:0/382811729 learned_addr learned my addr 172.21.15.92:0/382811729 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a02ffd700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a04101bc0 0x7f6a041033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a03fff700 1 -- 172.21.15.92:0/382811729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a04101bc0 msgr2=0x7f6a041033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a03fff700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a04101bc0 0x7f6a041033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a03fff700 1 -- 172.21.15.92:0/382811729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a04101200 msgr2=0x7f6a04102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a03fff700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a04101200 0x7f6a04102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a03fff700 1 -- 172.21.15.92:0/382811729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a041a6f60 con 0x7f6a04107240 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a02ffd700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a04101bc0 0x7f6a041033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a03fff700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 0x7f6a04103930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00ef30 tx=0x7f69fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a00ff9700 1 -- 172.21.15.92:0/382811729 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc0092e0 con 0x7f6a04107240 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a041a71f0 con 0x7f6a04107240 2024-03-19T01:52:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a00ff9700 1 -- 172.21.15.92:0/382811729 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69fc004bb0 con 0x7f6a04107240 2024-03-19T01:52:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a00ff9700 1 -- 172.21.15.92:0/382811729 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc004280 con 0x7f6a04107240 2024-03-19T01:52:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.394+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a041a7780 con 0x7f6a04107240 2024-03-19T01:52:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.396+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a04068fd0 con 0x7f6a04107240 2024-03-19T01:52:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.397+0000 7f6a00ff9700 1 -- 172.21.15.92:0/382811729 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f69fc01b030 con 0x7f6a04107240 2024-03-19T01:52:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.397+0000 7f6a00ff9700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f69ec0a2920 0x7f69ec0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.397+0000 7f6a037fe700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f69ec0a2920 0x7f69ec0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.397+0000 7f6a00ff9700 1 -- 172.21.15.92:0/382811729 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f69fc0ce5e0 con 0x7f6a04107240 2024-03-19T01:52:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.398+0000 7f6a037fe700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f69ec0a2920 0x7f69ec0a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69f400b2e0 tx=0x7f69f4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:03.404 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.402+0000 7f6a00ff9700 1 -- 172.21.15.92:0/382811729 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f69fc022020 con 0x7f6a04107240 2024-03-19T01:52:03.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.617+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6a041a79e0 con 0x7f6a04107240 2024-03-19T01:52:03.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.617+0000 7f6a00ff9700 1 -- 172.21.15.92:0/382811729 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f69fc010040 con 0x7f6a04107240 2024-03-19T01:52:03.618 INFO:teuthology.orchestra.run.smithi092.stdout:219043332145 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f69ec0a2920 msgr2=0x7f69ec0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f69ec0a2920 0x7f69ec0a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69f400b2e0 tx=0x7f69f4005960 comp rx=0 tx=0).stop 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 msgr2=0x7f6a04103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 0x7f6a04103930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00ef30 tx=0x7f69fc00c3a0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 shutdown_connections 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f69ec0a2920 0x7f69ec0a4de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a04107240 0x7f6a04103930 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a04101200 0x7f6a04102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 --2- 172.21.15.92:0/382811729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a04101bc0 0x7f6a041033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 >> 172.21.15.92:0/382811729 conn(0x7f6a040781b0 msgr2=0x7f6a041004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.619+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 shutdown_connections 2024-03-19T01:52:03.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.620+0000 7f6a09f6a700 1 -- 172.21.15.92:0/382811729 wait complete. 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 -- 172.21.15.92:0/3572082696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04641012c0 msgr2=0x7f046410f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 --2- 172.21.15.92:0/3572082696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04641012c0 0x7f046410f220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f045c00b0a0 tx=0x7f045c00b3b0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 -- 172.21.15.92:0/3572082696 shutdown_connections 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 --2- 172.21.15.92:0/3572082696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04641012c0 0x7f046410f220 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 --2- 172.21.15.92:0/3572082696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0464107380 0x7f0464107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 --2- 172.21.15.92:0/3572082696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0464100900 0x7f0464100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 -- 172.21.15.92:0/3572082696 >> 172.21.15.92:0/3572082696 conn(0x7f0464077810 msgr2=0x7f04640fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 -- 172.21.15.92:0/3572082696 shutdown_connections 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 -- 172.21.15.92:0/3572082696 wait complete. 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.628+0000 7f0469c4d700 1 Processor -- start 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0469c4d700 1 -- start start 2024-03-19T01:52:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0469c4d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0464100900 0x7f0464101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0469c4d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04641012c0 0x7f0464102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0469c4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0464107380 0x7f0464102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0469c4d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0464112740 con 0x7f0464100900 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0469c4d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04641125c0 con 0x7f0464107380 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0469c4d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04641128c0 con 0x7f04641012c0 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0462ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04641012c0 0x7f0464102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0462ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04641012c0 0x7f0464102020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:47136/0 (socket says 172.21.15.92:47136) 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0462ffd700 1 -- 172.21.15.92:0/802171807 learned_addr learned my addr 172.21.15.92:0/802171807 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0463fff700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0464107380 0x7f0464102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0462ffd700 1 -- 172.21.15.92:0/802171807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0464107380 msgr2=0x7f0464102560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0462ffd700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0464107380 0x7f0464102560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0462ffd700 1 -- 172.21.15.92:0/802171807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0464100900 msgr2=0x7f0464101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.629+0000 7f0462ffd700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0464100900 0x7f0464101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.630+0000 7f0462ffd700 1 -- 172.21.15.92:0/802171807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04641a70f0 con 0x7f04641012c0 2024-03-19T01:52:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.630+0000 7f0463fff700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0464107380 0x7f0464102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.630+0000 7f0462ffd700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04641012c0 0x7f0464102020 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f045800e9a0 tx=0x7f045800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.630+0000 7f0460ff9700 1 -- 172.21.15.92:0/802171807 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045800cbd0 con 0x7f04641012c0 2024-03-19T01:52:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.630+0000 7f0460ff9700 1 -- 172.21.15.92:0/802171807 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045800cd70 con 0x7f04641012c0 2024-03-19T01:52:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.630+0000 7f0469c4d700 1 -- 172.21.15.92:0/802171807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04641a73e0 con 0x7f04641012c0 2024-03-19T01:52:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.630+0000 7f0460ff9700 1 -- 172.21.15.92:0/802171807 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458018750 con 0x7f04641012c0 2024-03-19T01:52:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.630+0000 7f0469c4d700 1 -- 172.21.15.92:0/802171807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04641a7920 con 0x7f04641012c0 2024-03-19T01:52:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.631+0000 7f044a7fc700 1 -- 172.21.15.92:0/802171807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0444005330 con 0x7f04641012c0 2024-03-19T01:52:03.635 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.635+0000 7f0460ff9700 1 -- 172.21.15.92:0/802171807 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f0458010430 con 0x7f04641012c0 2024-03-19T01:52:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.635+0000 7f0460ff9700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f044c0a2920 0x7f044c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.635+0000 7f04637fe700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f044c0a2920 0x7f044c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.636+0000 7f0460ff9700 1 -- 172.21.15.92:0/802171807 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f04580cf350 con 0x7f04641012c0 2024-03-19T01:52:03.636 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.636+0000 7f04637fe700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f044c0a2920 0x7f044c0a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0454009b60 tx=0x7f0454009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:03.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.638+0000 7f0460ff9700 1 -- 172.21.15.92:0/802171807 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f045809c2a0 con 0x7f04641012c0 2024-03-19T01:52:03.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.928+0000 7f044a7fc700 1 -- 172.21.15.92:0/802171807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f04440058f0 con 0x7f04641012c0 2024-03-19T01:52:03.929 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:03 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/382811729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-19T01:52:03.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.928+0000 7f0460ff9700 1 -- 172.21.15.92:0/802171807 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f045809bba0 con 0x7f04641012c0 2024-03-19T01:52:03.929 INFO:teuthology.orchestra.run.smithi092.stdout:197568495670 2024-03-19T01:52:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.930+0000 7f044a7fc700 1 -- 172.21.15.92:0/802171807 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f044c0a2920 msgr2=0x7f044c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.930+0000 7f044a7fc700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f044c0a2920 0x7f044c0a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0454009b60 tx=0x7f0454009620 comp rx=0 tx=0).stop 2024-03-19T01:52:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.930+0000 7f044a7fc700 1 -- 172.21.15.92:0/802171807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04641012c0 msgr2=0x7f0464102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.930+0000 7f044a7fc700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04641012c0 0x7f0464102020 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f045800e9a0 tx=0x7f045800ecb0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.931+0000 7f044a7fc700 1 -- 172.21.15.92:0/802171807 shutdown_connections 2024-03-19T01:52:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.931+0000 7f044a7fc700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f044c0a2920 0x7f044c0a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.931+0000 7f044a7fc700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0464107380 0x7f0464102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.931+0000 7f044a7fc700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0464100900 0x7f0464101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.931+0000 7f044a7fc700 1 --2- 172.21.15.92:0/802171807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04641012c0 0x7f0464102020 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.931+0000 7f044a7fc700 1 -- 172.21.15.92:0/802171807 >> 172.21.15.92:0/802171807 conn(0x7f0464077810 msgr2=0x7f0464105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.931+0000 7f044a7fc700 1 -- 172.21.15.92:0/802171807 shutdown_connections 2024-03-19T01:52:03.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:03.931+0000 7f044a7fc700 1 -- 172.21.15.92:0/802171807 wait complete. 2024-03-19T01:52:03.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:03 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/382811729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-19T01:52:04.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:03 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/382811729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-19T01:52:04.555 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738451 for osd.0 2024-03-19T01:52:04.555 DEBUG:teuthology.parallel:result is None 2024-03-19T01:52:04.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:04 smithi092 ceph-mon[62574]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:04.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:04 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/802171807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-19T01:52:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:04 smithi159 ceph-mon[67110]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:04 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/802171807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-19T01:52:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:04 smithi144 ceph-mon[67199]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:04 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/802171807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-19T01:52:06.383 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332138 got 219043332145 for osd.7 2024-03-19T01:52:06.384 DEBUG:teuthology.parallel:result is None 2024-03-19T01:52:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:06 smithi159 ceph-mon[67110]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:07.044 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:06 smithi092 ceph-mon[62574]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:07.071 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495670 for osd.6 2024-03-19T01:52:07.071 DEBUG:teuthology.parallel:result is None 2024-03-19T01:52:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:06 smithi144 ceph-mon[67199]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 -- 172.21.15.92:0/3675790966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 msgr2=0x7fa85c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 --2- 172.21.15.92:0/3675790966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 0x7fa85c102810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009990 tx=0x7fa84c009ca0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 -- 172.21.15.92:0/3675790966 shutdown_connections 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 --2- 172.21.15.92:0/3675790966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c102d50 0x7fa85c10f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 --2- 172.21.15.92:0/3675790966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85c108470 0x7fa85c108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 --2- 172.21.15.92:0/3675790966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 0x7fa85c102810 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 -- 172.21.15.92:0/3675790966 >> 172.21.15.92:0/3675790966 conn(0x7fa85c0793d0 msgr2=0x7fa85c0797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 -- 172.21.15.92:0/3675790966 shutdown_connections 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 -- 172.21.15.92:0/3675790966 wait complete. 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 Processor -- start 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 -- start start 2024-03-19T01:52:07.284 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.282+0000 7fa864535700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 0x7fa85c074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa864535700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c102d50 0x7fa85c074b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa864535700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85c108470 0x7fa85c075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa864535700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1125f0 con 0x7fa85c102390 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa864535700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c112470 con 0x7fa85c108470 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa864535700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c112770 con 0x7fa85c102d50 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8622d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 0x7fa85c074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8622d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 0x7fa85c074610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59662/0 (socket says 172.21.15.92:59662) 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8622d1700 1 -- 172.21.15.92:0/2425281901 learned_addr learned my addr 172.21.15.92:0/2425281901 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa861ad0700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c102d50 0x7fa85c074b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa862ad2700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85c108470 0x7fa85c075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8622d1700 1 -- 172.21.15.92:0/2425281901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c102d50 msgr2=0x7fa85c074b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8622d1700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c102d50 0x7fa85c074b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8622d1700 1 -- 172.21.15.92:0/2425281901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85c108470 msgr2=0x7fa85c075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8622d1700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85c108470 0x7fa85c075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.285 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8622d1700 1 -- 172.21.15.92:0/2425281901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c1a6fd0 con 0x7fa85c102390 2024-03-19T01:52:07.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8622d1700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 0x7fa85c074610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa85800d980 tx=0x7fa85800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:07.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8537fe700 1 -- 172.21.15.92:0/2425281901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858016070 con 0x7fa85c102390 2024-03-19T01:52:07.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8537fe700 1 -- 172.21.15.92:0/2425281901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa858004d10 con 0x7fa85c102390 2024-03-19T01:52:07.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c1a72c0 con 0x7fa85c102390 2024-03-19T01:52:07.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa8537fe700 1 -- 172.21.15.92:0/2425281901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858005670 con 0x7fa85c102390 2024-03-19T01:52:07.286 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.283+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c1a77d0 con 0x7fa85c102390 2024-03-19T01:52:07.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.284+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c068fd0 con 0x7fa85c102390 2024-03-19T01:52:07.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.286+0000 7fa8537fe700 1 -- 172.21.15.92:0/2425281901 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fa858004750 con 0x7fa85c102390 2024-03-19T01:52:07.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.287+0000 7fa8537fe700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa8480a2920 0x7fa8480a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:07.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.287+0000 7fa861ad0700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa8480a2920 0x7fa8480a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:07.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.287+0000 7fa8537fe700 1 -- 172.21.15.92:0/2425281901 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa8580cee20 con 0x7fa85c102390 2024-03-19T01:52:07.291 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.287+0000 7fa861ad0700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa8480a2920 0x7fa8480a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00b2e0 tx=0x7fa84c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:07.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.292+0000 7fa8537fe700 1 -- 172.21.15.92:0/2425281901 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa85801f020 con 0x7fa85c102390 2024-03-19T01:52:07.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.519+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa85c02dbb0 con 0x7fa85c102390 2024-03-19T01:52:07.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.519+0000 7fa8537fe700 1 -- 172.21.15.92:0/2425281901 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa85809bd70 con 0x7fa85c102390 2024-03-19T01:52:07.520 INFO:teuthology.orchestra.run.smithi092.stdout:55834574928 2024-03-19T01:52:07.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.521+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa8480a2920 msgr2=0x7fa8480a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:07.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.521+0000 7fa864535700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa8480a2920 0x7fa8480a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00b2e0 tx=0x7fa84c005960 comp rx=0 tx=0).stop 2024-03-19T01:52:07.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.521+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 msgr2=0x7fa85c074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:07.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.521+0000 7fa864535700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 0x7fa85c074610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa85800d980 tx=0x7fa85800dc90 comp rx=0 tx=0).stop 2024-03-19T01:52:07.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.522+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 shutdown_connections 2024-03-19T01:52:07.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.522+0000 7fa864535700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa8480a2920 0x7fa8480a4de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.522+0000 7fa864535700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa85c108470 0x7fa85c075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.522+0000 7fa864535700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa85c102390 0x7fa85c074610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.522+0000 7fa864535700 1 --2- 172.21.15.92:0/2425281901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa85c102d50 0x7fa85c074b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:07.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.522+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 >> 172.21.15.92:0/2425281901 conn(0x7fa85c0793d0 msgr2=0x7fa85c101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:07.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.522+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 shutdown_connections 2024-03-19T01:52:07.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:07.522+0000 7fa864535700 1 -- 172.21.15.92:0/2425281901 wait complete. 2024-03-19T01:52:07.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:07 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2425281901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-19T01:52:07.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:07 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2425281901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-19T01:52:08.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:07 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2425281901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-19T01:52:08.305 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574928 for osd.1 2024-03-19T01:52:08.306 DEBUG:teuthology.parallel:result is None 2024-03-19T01:52:08.306 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-19T01:52:08.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-19T01:52:08.592 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:08.947 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:08 smithi092 ceph-mon[62574]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:08.947 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:52:08.947 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:52:08.947 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:52:08.947 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:52:08.947 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:52:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:08 smithi159 ceph-mon[67110]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:52:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:52:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:52:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:52:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:52:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:08 smithi144 ceph-mon[67199]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:52:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:52:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:52:09.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:52:09.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.333+0000 7f23e8027700 1 -- 172.21.15.92:0/2017241363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 msgr2=0x7f23e006aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 --2- 172.21.15.92:0/2017241363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 0x7f23e006aff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f23d80099c0 tx=0x7f23d8009cd0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 -- 172.21.15.92:0/2017241363 shutdown_connections 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 --2- 172.21.15.92:0/2017241363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e0107680 0x7f23e010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 --2- 172.21.15.92:0/2017241363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 0x7f23e006aff0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 --2- 172.21.15.92:0/2017241363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e006b5c0 0x7f23e0106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 -- 172.21.15.92:0/2017241363 >> 172.21.15.92:0/2017241363 conn(0x7f23e0078180 msgr2=0x7f23e0078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 -- 172.21.15.92:0/2017241363 shutdown_connections 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 -- 172.21.15.92:0/2017241363 wait complete. 2024-03-19T01:52:10.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 Processor -- start 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 -- start start 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 0x7f23e01012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e006b5c0 0x7f23e0101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.334+0000 7f23e8027700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e0107680 0x7f23e0101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e8027700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23e0112640 con 0x7f23e006ac10 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e8027700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23e01124c0 con 0x7f23e006b5c0 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e8027700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23e01127c0 con 0x7f23e0107680 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e5dc3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 0x7f23e01012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e5dc3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 0x7f23e01012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59682/0 (socket says 172.21.15.92:59682) 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e5dc3700 1 -- 172.21.15.92:0/3796637302 learned_addr learned my addr 172.21.15.92:0/3796637302 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e65c4700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e0107680 0x7f23e0101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e55c2700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e006b5c0 0x7f23e0101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e5dc3700 1 -- 172.21.15.92:0/3796637302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e0107680 msgr2=0x7f23e0101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e5dc3700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e0107680 0x7f23e0101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e5dc3700 1 -- 172.21.15.92:0/3796637302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e006b5c0 msgr2=0x7f23e0101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e5dc3700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e006b5c0 0x7f23e0101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e5dc3700 1 -- 172.21.15.92:0/3796637302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e01a7000 con 0x7f23e006ac10 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e5dc3700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 0x7f23e01012d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f23d80098e0 tx=0x7f23d800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23deffd700 1 -- 172.21.15.92:0/3796637302 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8009e60 con 0x7f23e006ac10 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23deffd700 1 -- 172.21.15.92:0/3796637302 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23d80050d0 con 0x7f23e006ac10 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e01a7290 con 0x7f23e006ac10 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23deffd700 1 -- 172.21.15.92:0/3796637302 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8003df0 con 0x7f23e006ac10 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.335+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e01a76e0 con 0x7f23e006ac10 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.336+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e010a830 con 0x7f23e006ac10 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.337+0000 7f23deffd700 1 -- 172.21.15.92:0/3796637302 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f23d8005280 con 0x7f23e006ac10 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.338+0000 7f23deffd700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f23c80a29a0 0x7f23c80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.338+0000 7f23e55c2700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f23c80a29a0 0x7f23c80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.338+0000 7f23deffd700 1 -- 172.21.15.92:0/3796637302 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f23d80cf940 con 0x7f23e006ac10 2024-03-19T01:52:10.350 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.338+0000 7f23e55c2700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f23c80a29a0 0x7f23c80a4e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23e0100db0 tx=0x7f23d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.344+0000 7f23deffd700 1 -- 172.21.15.92:0/3796637302 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f23d8013070 con 0x7f23e006ac10 2024-03-19T01:52:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.608+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23e0062750 con 0x7f23c80a29a0 2024-03-19T01:52:10.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.616+0000 7f23deffd700 1 -- 172.21.15.92:0/3796637302 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258975 (secure 0 0 0) 0x7f23e0062750 con 0x7f23c80a29a0 2024-03-19T01:52:10.620 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f23c80a29a0 msgr2=0x7f23c80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f23c80a29a0 0x7f23c80a4e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23e0100db0 tx=0x7f23d4009260 comp rx=0 tx=0).stop 2024-03-19T01:52:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 msgr2=0x7f23e01012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 0x7f23e01012d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f23d80098e0 tx=0x7f23d800f690 comp rx=0 tx=0).stop 2024-03-19T01:52:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 shutdown_connections 2024-03-19T01:52:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f23c80a29a0 0x7f23c80a4e60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e0107680 0x7f23e0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e006ac10 0x7f23e01012d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 --2- 172.21.15.92:0/3796637302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23e006b5c0 0x7f23e0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 >> 172.21.15.92:0/3796637302 conn(0x7f23e0078180 msgr2=0x7f23e00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 shutdown_connections 2024-03-19T01:52:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:10.619+0000 7f23e8027700 1 -- 172.21.15.92:0/3796637302 wait complete. 2024-03-19T01:52:10.623 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-03-19T01:52:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:10 smithi159 ceph-mon[67110]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:10.986 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:10 smithi092 ceph-mon[62574]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:10 smithi144 ceph-mon[67199]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:11.355 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-03-19T01:52:09.475964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656880,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":884,"num_read_kb":562,"num_write":419,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1181503,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352608,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589216,"statfs":{"total":1151940427776,"available":1151579357184,"internally_reserved":0,"allocated":11980800,"data_stored":7437924,"data_compressed":112575,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19140,"internal_metadata":348894524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003172"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588689+0000","last_change":"2024-03-19T01:50:50.561781+0000","last_active":"2024-03-19T01:51:40.588689+0000","last_peered":"2024-03-19T01:51:40.588689+0000","last_clean":"2024-03-19T01:51:40.588689+0000","last_became_active":"2024-03-19T01:50:50.561402+0000","last_became_peered":"2024-03-19T01:50:50.561402+0000","last_unstale":"2024-03-19T01:51:40.588689+0000","last_undegraded":"2024-03-19T01:51:40.588689+0000","last_fullsized":"2024-03-19T01:51:40.588689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:07.257394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584639+0000","last_change":"2024-03-19T01:50:50.558625+0000","last_active":"2024-03-19T01:51:40.584639+0000","last_peered":"2024-03-19T01:51:40.584639+0000","last_clean":"2024-03-19T01:51:40.584639+0000","last_became_active":"2024-03-19T01:50:50.557349+0000","last_became_peered":"2024-03-19T01:50:50.557349+0000","last_unstale":"2024-03-19T01:51:40.584639+0000","last_undegraded":"2024-03-19T01:51:40.584639+0000","last_fullsized":"2024-03-19T01:51:40.584639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:05.791529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584544+0000","last_change":"2024-03-19T01:50:50.554103+0000","last_active":"2024-03-19T01:51:40.584544+0000","last_peered":"2024-03-19T01:51:40.584544+0000","last_clean":"2024-03-19T01:51:40.584544+0000","last_became_active":"2024-03-19T01:50:50.553734+0000","last_became_peered":"2024-03-19T01:50:50.553734+0000","last_unstale":"2024-03-19T01:51:40.584544+0000","last_undegraded":"2024-03-19T01:51:40.584544+0000","last_fullsized":"2024-03-19T01:51:40.584544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:18:02.768774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.852556+0000","last_change":"2024-03-19T01:50:50.571980+0000","last_active":"2024-03-19T01:51:40.852556+0000","last_peered":"2024-03-19T01:51:40.852556+0000","last_clean":"2024-03-19T01:51:40.852556+0000","last_became_active":"2024-03-19T01:50:50.571679+0000","last_became_peered":"2024-03-19T01:50:50.571679+0000","last_unstale":"2024-03-19T01:51:40.852556+0000","last_undegraded":"2024-03-19T01:51:40.852556+0000","last_fullsized":"2024-03-19T01:51:40.852556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:30:08.946509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585703+0000","last_change":"2024-03-19T01:50:50.563314+0000","last_active":"2024-03-19T01:51:40.585703+0000","last_peered":"2024-03-19T01:51:40.585703+0000","last_clean":"2024-03-19T01:51:40.585703+0000","last_became_active":"2024-03-19T01:50:50.562953+0000","last_became_peered":"2024-03-19T01:50:50.562953+0000","last_unstale":"2024-03-19T01:51:40.585703+0000","last_undegraded":"2024-03-19T01:51:40.585703+0000","last_fullsized":"2024-03-19T01:51:40.585703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:38.139670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.589887+0000","last_change":"2024-03-19T01:50:50.555305+0000","last_active":"2024-03-19T01:51:40.589887+0000","last_peered":"2024-03-19T01:51:40.589887+0000","last_clean":"2024-03-19T01:51:40.589887+0000","last_became_active":"2024-03-19T01:50:50.554930+0000","last_became_peered":"2024-03-19T01:50:50.554930+0000","last_unstale":"2024-03-19T01:51:40.589887+0000","last_undegraded":"2024-03-19T01:51:40.589887+0000","last_fullsized":"2024-03-19T01:51:40.589887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:05:03.418572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584773+0000","last_change":"2024-03-19T01:50:50.557381+0000","last_active":"2024-03-19T01:51:40.584773+0000","last_peered":"2024-03-19T01:51:40.584773+0000","last_clean":"2024-03-19T01:51:40.584773+0000","last_became_active":"2024-03-19T01:50:50.556804+0000","last_became_peered":"2024-03-19T01:50:50.556804+0000","last_unstale":"2024-03-19T01:51:40.584773+0000","last_undegraded":"2024-03-19T01:51:40.584773+0000","last_fullsized":"2024-03-19T01:51:40.584773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:54:00.676606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.591431+0000","last_change":"2024-03-19T01:51:40.591005+0000","last_active":"2024-03-19T01:51:40.591431+0000","last_peered":"2024-03-19T01:51:40.591431+0000","last_clean":"2024-03-19T01:51:40.591431+0000","last_became_active":"2024-03-19T01:51:40.590558+0000","last_became_peered":"2024-03-19T01:51:40.590558+0000","last_unstale":"2024-03-19T01:51:40.591431+0000","last_undegraded":"2024-03-19T01:51:40.591431+0000","last_fullsized":"2024-03-19T01:51:40.591431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:19:45.880448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590704+0000","last_change":"2024-03-19T01:50:50.562970+0000","last_active":"2024-03-19T01:51:40.590704+0000","last_peered":"2024-03-19T01:51:40.590704+0000","last_clean":"2024-03-19T01:51:40.590704+0000","last_became_active":"2024-03-19T01:50:50.562372+0000","last_became_peered":"2024-03-19T01:50:50.562372+0000","last_unstale":"2024-03-19T01:51:40.590704+0000","last_undegraded":"2024-03-19T01:51:40.590704+0000","last_fullsized":"2024-03-19T01:51:40.590704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:57.861083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590600+0000","last_change":"2024-03-19T01:50:50.559369+0000","last_active":"2024-03-19T01:51:40.590600+0000","last_peered":"2024-03-19T01:51:40.590600+0000","last_clean":"2024-03-19T01:51:40.590600+0000","last_became_active":"2024-03-19T01:50:50.559062+0000","last_became_peered":"2024-03-19T01:50:50.559062+0000","last_unstale":"2024-03-19T01:51:40.590600+0000","last_undegraded":"2024-03-19T01:51:40.590600+0000","last_fullsized":"2024-03-19T01:51:40.590600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:08:22.977566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585131+0000","last_change":"2024-03-19T01:50:50.567000+0000","last_active":"2024-03-19T01:51:40.585131+0000","last_peered":"2024-03-19T01:51:40.585131+0000","last_clean":"2024-03-19T01:51:40.585131+0000","last_became_active":"2024-03-19T01:50:50.566613+0000","last_became_peered":"2024-03-19T01:50:50.566613+0000","last_unstale":"2024-03-19T01:51:40.585131+0000","last_undegraded":"2024-03-19T01:51:40.585131+0000","last_fullsized":"2024-03-19T01:51:40.585131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:40:39.132559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.852554+0000","last_change":"2024-03-19T01:50:50.564354+0000","last_active":"2024-03-19T01:51:40.852554+0000","last_peered":"2024-03-19T01:51:40.852554+0000","last_clean":"2024-03-19T01:51:40.852554+0000","last_became_active":"2024-03-19T01:50:50.563997+0000","last_became_peered":"2024-03-19T01:50:50.563997+0000","last_unstale":"2024-03-19T01:51:40.852554+0000","last_undegraded":"2024-03-19T01:51:40.852554+0000","last_fullsized":"2024-03-19T01:51:40.852554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:27:50.266867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705190+0000","last_change":"2024-03-19T01:50:50.539164+0000","last_active":"2024-03-19T01:51:40.705190+0000","last_peered":"2024-03-19T01:51:40.705190+0000","last_clean":"2024-03-19T01:51:40.705190+0000","last_became_active":"2024-03-19T01:50:50.538792+0000","last_became_peered":"2024-03-19T01:50:50.538792+0000","last_unstale":"2024-03-19T01:51:40.705190+0000","last_undegraded":"2024-03-19T01:51:40.705190+0000","last_fullsized":"2024-03-19T01:51:40.705190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:07:56.700994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584286+0000","last_change":"2024-03-19T01:50:50.567590+0000","last_active":"2024-03-19T01:51:40.584286+0000","last_peered":"2024-03-19T01:51:40.584286+0000","last_clean":"2024-03-19T01:51:40.584286+0000","last_became_active":"2024-03-19T01:50:50.567225+0000","last_became_peered":"2024-03-19T01:50:50.567225+0000","last_unstale":"2024-03-19T01:51:40.584286+0000","last_undegraded":"2024-03-19T01:51:40.584286+0000","last_fullsized":"2024-03-19T01:51:40.584286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:55.537567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.592111+0000","last_change":"2024-03-19T01:51:40.592111+0000","last_active":"2024-03-19T01:51:40.592111+0000","last_peered":"2024-03-19T01:51:40.592111+0000","last_clean":"2024-03-19T01:51:40.592111+0000","last_became_active":"2024-03-19T01:51:40.591725+0000","last_became_peered":"2024-03-19T01:51:40.591725+0000","last_unstale":"2024-03-19T01:51:40.592111+0000","last_undegraded":"2024-03-19T01:51:40.592111+0000","last_fullsized":"2024-03-19T01:51:40.592111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:05:36.932918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.589971+0000","last_change":"2024-03-19T01:50:50.558616+0000","last_active":"2024-03-19T01:51:40.589971+0000","last_peered":"2024-03-19T01:51:40.589971+0000","last_clean":"2024-03-19T01:51:40.589971+0000","last_became_active":"2024-03-19T01:50:50.558173+0000","last_became_peered":"2024-03-19T01:50:50.558173+0000","last_unstale":"2024-03-19T01:51:40.589971+0000","last_undegraded":"2024-03-19T01:51:40.589971+0000","last_fullsized":"2024-03-19T01:51:40.589971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:04:47.174828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588788+0000","last_change":"2024-03-19T01:50:50.563450+0000","last_active":"2024-03-19T01:51:40.588788+0000","last_peered":"2024-03-19T01:51:40.588788+0000","last_clean":"2024-03-19T01:51:40.588788+0000","last_became_active":"2024-03-19T01:50:50.562663+0000","last_became_peered":"2024-03-19T01:50:50.562663+0000","last_unstale":"2024-03-19T01:51:40.588788+0000","last_undegraded":"2024-03-19T01:51:40.588788+0000","last_fullsized":"2024-03-19T01:51:40.588788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:57:47.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588262+0000","last_change":"2024-03-19T01:50:50.559848+0000","last_active":"2024-03-19T01:51:40.588262+0000","last_peered":"2024-03-19T01:51:40.588262+0000","last_clean":"2024-03-19T01:51:40.588262+0000","last_became_active":"2024-03-19T01:50:50.559515+0000","last_became_peered":"2024-03-19T01:50:50.559515+0000","last_unstale":"2024-03-19T01:51:40.588262+0000","last_undegraded":"2024-03-19T01:51:40.588262+0000","last_fullsized":"2024-03-19T01:51:40.588262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:16:15.262613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590368+0000","last_change":"2024-03-19T01:50:50.559994+0000","last_active":"2024-03-19T01:51:40.590368+0000","last_peered":"2024-03-19T01:51:40.590368+0000","last_clean":"2024-03-19T01:51:40.590368+0000","last_became_active":"2024-03-19T01:50:50.558759+0000","last_became_peered":"2024-03-19T01:50:50.558759+0000","last_unstale":"2024-03-19T01:51:40.590368+0000","last_undegraded":"2024-03-19T01:51:40.590368+0000","last_fullsized":"2024-03-19T01:51:40.590368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:53.957096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:05.051671+0000","last_change":"2024-03-19T01:50:49.550130+0000","last_active":"2024-03-19T01:52:05.051671+0000","last_peered":"2024-03-19T01:52:05.051671+0000","last_clean":"2024-03-19T01:52:05.051671+0000","last_became_active":"2024-03-19T01:50:49.549374+0000","last_became_peered":"2024-03-19T01:50:49.549374+0000","last_unstale":"2024-03-19T01:52:05.051671+0000","last_undegraded":"2024-03-19T01:52:05.051671+0000","last_fullsized":"2024-03-19T01:52:05.051671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:14:43.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588755+0000","last_change":"2024-03-19T01:50:50.570319+0000","last_active":"2024-03-19T01:51:40.588755+0000","last_peered":"2024-03-19T01:51:40.588755+0000","last_clean":"2024-03-19T01:51:40.588755+0000","last_became_active":"2024-03-19T01:50:50.569937+0000","last_became_peered":"2024-03-19T01:50:50.569937+0000","last_unstale":"2024-03-19T01:51:40.588755+0000","last_undegraded":"2024-03-19T01:51:40.588755+0000","last_fullsized":"2024-03-19T01:51:40.588755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:38:00.024688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588415+0000","last_change":"2024-03-19T01:50:49.551261+0000","last_active":"2024-03-19T01:51:40.588415+0000","last_peered":"2024-03-19T01:51:40.588415+0000","last_clean":"2024-03-19T01:51:40.588415+0000","last_became_active":"2024-03-19T01:50:49.550949+0000","last_became_peered":"2024-03-19T01:50:49.550949+0000","last_unstale":"2024-03-19T01:51:40.588415+0000","last_undegraded":"2024-03-19T01:51:40.588415+0000","last_fullsized":"2024-03-19T01:51:40.588415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:23:49.064155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584912+0000","last_change":"2024-03-19T01:50:50.554303+0000","last_active":"2024-03-19T01:51:40.584912+0000","last_peered":"2024-03-19T01:51:40.584912+0000","last_clean":"2024-03-19T01:51:40.584912+0000","last_became_active":"2024-03-19T01:50:50.553765+0000","last_became_peered":"2024-03-19T01:50:50.553765+0000","last_unstale":"2024-03-19T01:51:40.584912+0000","last_undegraded":"2024-03-19T01:51:40.584912+0000","last_fullsized":"2024-03-19T01:51:40.584912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:42.246266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.586323+0000","last_change":"2024-03-19T01:50:49.554688+0000","last_active":"2024-03-19T01:51:40.586323+0000","last_peered":"2024-03-19T01:51:40.586323+0000","last_clean":"2024-03-19T01:51:40.586323+0000","last_became_active":"2024-03-19T01:50:49.554289+0000","last_became_peered":"2024-03-19T01:50:49.554289+0000","last_unstale":"2024-03-19T01:51:40.586323+0000","last_undegraded":"2024-03-19T01:51:40.586323+0000","last_fullsized":"2024-03-19T01:51:40.586323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:13.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590266+0000","last_change":"2024-03-19T01:50:50.563305+0000","last_active":"2024-03-19T01:51:40.590266+0000","last_peered":"2024-03-19T01:51:40.590266+0000","last_clean":"2024-03-19T01:51:40.590266+0000","last_became_active":"2024-03-19T01:50:50.561819+0000","last_became_peered":"2024-03-19T01:50:50.561819+0000","last_unstale":"2024-03-19T01:51:40.590266+0000","last_undegraded":"2024-03-19T01:51:40.590266+0000","last_fullsized":"2024-03-19T01:51:40.590266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:11:56.175649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.599378+0000","last_change":"2024-03-19T01:51:40.599378+0000","last_active":"2024-03-19T01:51:40.599378+0000","last_peered":"2024-03-19T01:51:40.599378+0000","last_clean":"2024-03-19T01:51:40.599378+0000","last_became_active":"2024-03-19T01:51:40.592404+0000","last_became_peered":"2024-03-19T01:51:40.592404+0000","last_unstale":"2024-03-19T01:51:40.599378+0000","last_undegraded":"2024-03-19T01:51:40.599378+0000","last_fullsized":"2024-03-19T01:51:40.599378+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:03:37.485954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587303+0000","last_change":"2024-03-19T01:50:50.560178+0000","last_active":"2024-03-19T01:51:40.587303+0000","last_peered":"2024-03-19T01:51:40.587303+0000","last_clean":"2024-03-19T01:51:40.587303+0000","last_became_active":"2024-03-19T01:50:50.559853+0000","last_became_peered":"2024-03-19T01:50:50.559853+0000","last_unstale":"2024-03-19T01:51:40.587303+0000","last_undegraded":"2024-03-19T01:51:40.587303+0000","last_fullsized":"2024-03-19T01:51:40.587303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:10:04.812207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:07.044897+0000","last_change":"2024-03-19T01:50:49.550194+0000","last_active":"2024-03-19T01:52:07.044897+0000","last_peered":"2024-03-19T01:52:07.044897+0000","last_clean":"2024-03-19T01:52:07.044897+0000","last_became_active":"2024-03-19T01:50:49.549801+0000","last_became_peered":"2024-03-19T01:50:49.549801+0000","last_unstale":"2024-03-19T01:52:07.044897+0000","last_undegraded":"2024-03-19T01:52:07.044897+0000","last_fullsized":"2024-03-19T01:52:07.044897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:43.110159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585616+0000","last_change":"2024-03-19T01:50:50.560913+0000","last_active":"2024-03-19T01:51:40.585616+0000","last_peered":"2024-03-19T01:51:40.585616+0000","last_clean":"2024-03-19T01:51:40.585616+0000","last_became_active":"2024-03-19T01:50:50.559210+0000","last_became_peered":"2024-03-19T01:50:50.559210+0000","last_unstale":"2024-03-19T01:51:40.585616+0000","last_undegraded":"2024-03-19T01:51:40.585616+0000","last_fullsized":"2024-03-19T01:51:40.585616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:14.814374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.589771+0000","last_change":"2024-03-19T01:50:49.548778+0000","last_active":"2024-03-19T01:51:40.589771+0000","last_peered":"2024-03-19T01:51:40.589771+0000","last_clean":"2024-03-19T01:51:40.589771+0000","last_became_active":"2024-03-19T01:50:49.548414+0000","last_became_peered":"2024-03-19T01:50:49.548414+0000","last_unstale":"2024-03-19T01:51:40.589771+0000","last_undegraded":"2024-03-19T01:51:40.589771+0000","last_fullsized":"2024-03-19T01:51:40.589771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:03:37.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587617+0000","last_change":"2024-03-19T01:50:50.559242+0000","last_active":"2024-03-19T01:51:40.587617+0000","last_peered":"2024-03-19T01:51:40.587617+0000","last_clean":"2024-03-19T01:51:40.587617+0000","last_became_active":"2024-03-19T01:50:50.558904+0000","last_became_peered":"2024-03-19T01:50:50.558904+0000","last_unstale":"2024-03-19T01:51:40.587617+0000","last_undegraded":"2024-03-19T01:51:40.587617+0000","last_fullsized":"2024-03-19T01:51:40.587617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:59.249180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.704875+0000","last_change":"2024-03-19T01:50:49.548331+0000","last_active":"2024-03-19T01:51:40.704875+0000","last_peered":"2024-03-19T01:51:40.704875+0000","last_clean":"2024-03-19T01:51:40.704875+0000","last_became_active":"2024-03-19T01:50:49.547993+0000","last_became_peered":"2024-03-19T01:50:49.547993+0000","last_unstale":"2024-03-19T01:51:40.704875+0000","last_undegraded":"2024-03-19T01:51:40.704875+0000","last_fullsized":"2024-03-19T01:51:40.704875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:30.059133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.586980+0000","last_change":"2024-03-19T01:50:50.553166+0000","last_active":"2024-03-19T01:51:40.586980+0000","last_peered":"2024-03-19T01:51:40.586980+0000","last_clean":"2024-03-19T01:51:40.586980+0000","last_became_active":"2024-03-19T01:50:50.552520+0000","last_became_peered":"2024-03-19T01:50:50.552520+0000","last_unstale":"2024-03-19T01:51:40.586980+0000","last_undegraded":"2024-03-19T01:51:40.586980+0000","last_fullsized":"2024-03-19T01:51:40.586980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:10:24.515191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587095+0000","last_change":"2024-03-19T01:50:49.548868+0000","last_active":"2024-03-19T01:51:40.587095+0000","last_peered":"2024-03-19T01:51:40.587095+0000","last_clean":"2024-03-19T01:51:40.587095+0000","last_became_active":"2024-03-19T01:50:49.548548+0000","last_became_peered":"2024-03-19T01:50:49.548548+0000","last_unstale":"2024-03-19T01:51:40.587095+0000","last_undegraded":"2024-03-19T01:51:40.587095+0000","last_fullsized":"2024-03-19T01:51:40.587095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:21:36.920067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587869+0000","last_change":"2024-03-19T01:50:50.559870+0000","last_active":"2024-03-19T01:51:40.587869+0000","last_peered":"2024-03-19T01:51:40.587869+0000","last_clean":"2024-03-19T01:51:40.587869+0000","last_became_active":"2024-03-19T01:50:50.559530+0000","last_became_peered":"2024-03-19T01:50:50.559530+0000","last_unstale":"2024-03-19T01:51:40.587869+0000","last_undegraded":"2024-03-19T01:51:40.587869+0000","last_fullsized":"2024-03-19T01:51:40.587869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:39:48.635416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":169,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:08.045125+0000","last_change":"2024-03-19T01:50:49.554733+0000","last_active":"2024-03-19T01:52:08.045125+0000","last_peered":"2024-03-19T01:52:08.045125+0000","last_clean":"2024-03-19T01:52:08.045125+0000","last_became_active":"2024-03-19T01:50:49.554366+0000","last_became_peered":"2024-03-19T01:50:49.554366+0000","last_unstale":"2024-03-19T01:52:08.045125+0000","last_undegraded":"2024-03-19T01:52:08.045125+0000","last_fullsized":"2024-03-19T01:52:08.045125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:53:03.790255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590608+0000","last_change":"2024-03-19T01:50:50.561526+0000","last_active":"2024-03-19T01:51:40.590608+0000","last_peered":"2024-03-19T01:51:40.590608+0000","last_clean":"2024-03-19T01:51:40.590608+0000","last_became_active":"2024-03-19T01:50:50.561189+0000","last_became_peered":"2024-03-19T01:50:50.561189+0000","last_unstale":"2024-03-19T01:51:40.590608+0000","last_undegraded":"2024-03-19T01:51:40.590608+0000","last_fullsized":"2024-03-19T01:51:40.590608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:15:18.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585125+0000","last_change":"2024-03-19T01:50:49.553701+0000","last_active":"2024-03-19T01:51:40.585125+0000","last_peered":"2024-03-19T01:51:40.585125+0000","last_clean":"2024-03-19T01:51:40.585125+0000","last_became_active":"2024-03-19T01:50:49.553079+0000","last_became_peered":"2024-03-19T01:50:49.553079+0000","last_unstale":"2024-03-19T01:51:40.585125+0000","last_undegraded":"2024-03-19T01:51:40.585125+0000","last_fullsized":"2024-03-19T01:51:40.585125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:19.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.706526+0000","last_change":"2024-03-19T01:51:40.706526+0000","last_active":"2024-03-19T01:51:40.706526+0000","last_peered":"2024-03-19T01:51:40.706526+0000","last_clean":"2024-03-19T01:51:40.706526+0000","last_became_active":"2024-03-19T01:51:40.706109+0000","last_became_peered":"2024-03-19T01:51:40.706109+0000","last_unstale":"2024-03-19T01:51:40.706526+0000","last_undegraded":"2024-03-19T01:51:40.706526+0000","last_fullsized":"2024-03-19T01:51:40.706526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:14:17.713288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587354+0000","last_change":"2024-03-19T01:50:49.553573+0000","last_active":"2024-03-19T01:51:40.587354+0000","last_peered":"2024-03-19T01:51:40.587354+0000","last_clean":"2024-03-19T01:51:40.587354+0000","last_became_active":"2024-03-19T01:50:49.553209+0000","last_became_peered":"2024-03-19T01:50:49.553209+0000","last_unstale":"2024-03-19T01:51:40.587354+0000","last_undegraded":"2024-03-19T01:51:40.587354+0000","last_fullsized":"2024-03-19T01:51:40.587354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:47:43.235323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590313+0000","last_change":"2024-03-19T01:50:50.568133+0000","last_active":"2024-03-19T01:51:40.590313+0000","last_peered":"2024-03-19T01:51:40.590313+0000","last_clean":"2024-03-19T01:51:40.590313+0000","last_became_active":"2024-03-19T01:50:50.567783+0000","last_became_peered":"2024-03-19T01:50:50.567783+0000","last_unstale":"2024-03-19T01:51:40.590313+0000","last_undegraded":"2024-03-19T01:51:40.590313+0000","last_fullsized":"2024-03-19T01:51:40.590313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:16.077111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":201,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:08.045117+0000","last_change":"2024-03-19T01:50:49.546463+0000","last_active":"2024-03-19T01:52:08.045117+0000","last_peered":"2024-03-19T01:52:08.045117+0000","last_clean":"2024-03-19T01:52:08.045117+0000","last_became_active":"2024-03-19T01:50:49.546033+0000","last_became_peered":"2024-03-19T01:50:49.546033+0000","last_unstale":"2024-03-19T01:52:08.045117+0000","last_undegraded":"2024-03-19T01:52:08.045117+0000","last_fullsized":"2024-03-19T01:52:08.045117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:29:13.933301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":153,"num_read_kb":135,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.586113+0000","last_change":"2024-03-19T01:50:50.557576+0000","last_active":"2024-03-19T01:51:40.586113+0000","last_peered":"2024-03-19T01:51:40.586113+0000","last_clean":"2024-03-19T01:51:40.586113+0000","last_became_active":"2024-03-19T01:50:50.555024+0000","last_became_peered":"2024-03-19T01:50:50.555024+0000","last_unstale":"2024-03-19T01:51:40.586113+0000","last_undegraded":"2024-03-19T01:51:40.586113+0000","last_fullsized":"2024-03-19T01:51:40.586113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:17.252079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.608019+0000","last_change":"2024-03-19T01:51:40.608019+0000","last_active":"2024-03-19T01:51:40.608019+0000","last_peered":"2024-03-19T01:51:40.608019+0000","last_clean":"2024-03-19T01:51:40.608019+0000","last_became_active":"2024-03-19T01:51:40.592774+0000","last_became_peered":"2024-03-19T01:51:40.592774+0000","last_unstale":"2024-03-19T01:51:40.608019+0000","last_undegraded":"2024-03-19T01:51:40.608019+0000","last_fullsized":"2024-03-19T01:51:40.608019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:47.204951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.704972+0000","last_change":"2024-03-19T01:50:50.554679+0000","last_active":"2024-03-19T01:51:40.704972+0000","last_peered":"2024-03-19T01:51:40.704972+0000","last_clean":"2024-03-19T01:51:40.704972+0000","last_became_active":"2024-03-19T01:50:50.554361+0000","last_became_peered":"2024-03-19T01:50:50.554361+0000","last_unstale":"2024-03-19T01:51:40.704972+0000","last_undegraded":"2024-03-19T01:51:40.704972+0000","last_fullsized":"2024-03-19T01:51:40.704972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:44:53.385907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587276+0000","last_change":"2024-03-19T01:50:49.547590+0000","last_active":"2024-03-19T01:51:40.587276+0000","last_peered":"2024-03-19T01:51:40.587276+0000","last_clean":"2024-03-19T01:51:40.587276+0000","last_became_active":"2024-03-19T01:50:49.547261+0000","last_became_peered":"2024-03-19T01:50:49.547261+0000","last_unstale":"2024-03-19T01:51:40.587276+0000","last_undegraded":"2024-03-19T01:51:40.587276+0000","last_fullsized":"2024-03-19T01:51:40.587276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:47.353161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585043+0000","last_change":"2024-03-19T01:50:50.554830+0000","last_active":"2024-03-19T01:51:40.585043+0000","last_peered":"2024-03-19T01:51:40.585043+0000","last_clean":"2024-03-19T01:51:40.585043+0000","last_became_active":"2024-03-19T01:50:50.554495+0000","last_became_peered":"2024-03-19T01:50:50.554495+0000","last_unstale":"2024-03-19T01:51:40.585043+0000","last_undegraded":"2024-03-19T01:51:40.585043+0000","last_fullsized":"2024-03-19T01:51:40.585043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:44.098279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":99,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:07.044601+0000","last_change":"2024-03-19T01:50:49.550056+0000","last_active":"2024-03-19T01:52:07.044601+0000","last_peered":"2024-03-19T01:52:07.044601+0000","last_clean":"2024-03-19T01:52:07.044601+0000","last_became_active":"2024-03-19T01:50:49.549404+0000","last_became_peered":"2024-03-19T01:50:49.549404+0000","last_unstale":"2024-03-19T01:52:07.044601+0000","last_undegraded":"2024-03-19T01:52:07.044601+0000","last_fullsized":"2024-03-19T01:52:07.044601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:38:38.656100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584735+0000","last_change":"2024-03-19T01:50:50.559443+0000","last_active":"2024-03-19T01:51:40.584735+0000","last_peered":"2024-03-19T01:51:40.584735+0000","last_clean":"2024-03-19T01:51:40.584735+0000","last_became_active":"2024-03-19T01:50:50.558992+0000","last_became_peered":"2024-03-19T01:50:50.558992+0000","last_unstale":"2024-03-19T01:51:40.584735+0000","last_undegraded":"2024-03-19T01:51:40.584735+0000","last_fullsized":"2024-03-19T01:51:40.584735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:06:52.144816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585516+0000","last_change":"2024-03-19T01:50:49.548423+0000","last_active":"2024-03-19T01:51:40.585516+0000","last_peered":"2024-03-19T01:51:40.585516+0000","last_clean":"2024-03-19T01:51:40.585516+0000","last_became_active":"2024-03-19T01:50:49.548071+0000","last_became_peered":"2024-03-19T01:50:49.548071+0000","last_unstale":"2024-03-19T01:51:40.585516+0000","last_undegraded":"2024-03-19T01:51:40.585516+0000","last_fullsized":"2024-03-19T01:51:40.585516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:06:00.789358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587258+0000","last_change":"2024-03-19T01:50:50.558569+0000","last_active":"2024-03-19T01:51:40.587258+0000","last_peered":"2024-03-19T01:51:40.587258+0000","last_clean":"2024-03-19T01:51:40.587258+0000","last_became_active":"2024-03-19T01:50:50.557991+0000","last_became_peered":"2024-03-19T01:50:50.557991+0000","last_unstale":"2024-03-19T01:51:40.587258+0000","last_undegraded":"2024-03-19T01:51:40.587258+0000","last_fullsized":"2024-03-19T01:51:40.587258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:55:07.199605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.586157+0000","last_change":"2024-03-19T01:50:49.552491+0000","last_active":"2024-03-19T01:51:40.586157+0000","last_peered":"2024-03-19T01:51:40.586157+0000","last_clean":"2024-03-19T01:51:40.586157+0000","last_became_active":"2024-03-19T01:50:49.552133+0000","last_became_peered":"2024-03-19T01:50:49.552133+0000","last_unstale":"2024-03-19T01:51:40.586157+0000","last_undegraded":"2024-03-19T01:51:40.586157+0000","last_fullsized":"2024-03-19T01:51:40.586157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:44.458968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590244+0000","last_change":"2024-03-19T01:51:40.590244+0000","last_active":"2024-03-19T01:51:40.590244+0000","last_peered":"2024-03-19T01:51:40.590244+0000","last_clean":"2024-03-19T01:51:40.590244+0000","last_became_active":"2024-03-19T01:51:40.589898+0000","last_became_peered":"2024-03-19T01:51:40.589898+0000","last_unstale":"2024-03-19T01:51:40.590244+0000","last_undegraded":"2024-03-19T01:51:40.590244+0000","last_fullsized":"2024-03-19T01:51:40.590244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:16:30.244828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587559+0000","last_change":"2024-03-19T01:50:49.549586+0000","last_active":"2024-03-19T01:51:40.587559+0000","last_peered":"2024-03-19T01:51:40.587559+0000","last_clean":"2024-03-19T01:51:40.587559+0000","last_became_active":"2024-03-19T01:50:49.549224+0000","last_became_peered":"2024-03-19T01:50:49.549224+0000","last_unstale":"2024-03-19T01:51:40.587559+0000","last_undegraded":"2024-03-19T01:51:40.587559+0000","last_fullsized":"2024-03-19T01:51:40.587559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:24:30.591771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590250+0000","last_change":"2024-03-19T01:50:50.554120+0000","last_active":"2024-03-19T01:51:40.590250+0000","last_peered":"2024-03-19T01:51:40.590250+0000","last_clean":"2024-03-19T01:51:40.590250+0000","last_became_active":"2024-03-19T01:50:50.553636+0000","last_became_peered":"2024-03-19T01:50:50.553636+0000","last_unstale":"2024-03-19T01:51:40.590250+0000","last_undegraded":"2024-03-19T01:51:40.590250+0000","last_fullsized":"2024-03-19T01:51:40.590250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:27.634688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:04.051052+0000","last_change":"2024-03-19T01:50:49.551532+0000","last_active":"2024-03-19T01:52:04.051052+0000","last_peered":"2024-03-19T01:52:04.051052+0000","last_clean":"2024-03-19T01:52:04.051052+0000","last_became_active":"2024-03-19T01:50:49.551060+0000","last_became_peered":"2024-03-19T01:50:49.551060+0000","last_unstale":"2024-03-19T01:52:04.051052+0000","last_undegraded":"2024-03-19T01:52:04.051052+0000","last_fullsized":"2024-03-19T01:52:04.051052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:00.339785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590459+0000","last_change":"2024-03-19T01:50:50.556606+0000","last_active":"2024-03-19T01:51:40.590459+0000","last_peered":"2024-03-19T01:51:40.590459+0000","last_clean":"2024-03-19T01:51:40.590459+0000","last_became_active":"2024-03-19T01:50:50.556089+0000","last_became_peered":"2024-03-19T01:50:50.556089+0000","last_unstale":"2024-03-19T01:51:40.590459+0000","last_undegraded":"2024-03-19T01:51:40.590459+0000","last_fullsized":"2024-03-19T01:51:40.590459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:24.256326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587306+0000","last_change":"2024-03-19T01:50:50.564926+0000","last_active":"2024-03-19T01:51:40.587306+0000","last_peered":"2024-03-19T01:51:40.587306+0000","last_clean":"2024-03-19T01:51:40.587306+0000","last_became_active":"2024-03-19T01:50:50.564612+0000","last_became_peered":"2024-03-19T01:50:50.564612+0000","last_unstale":"2024-03-19T01:51:40.587306+0000","last_undegraded":"2024-03-19T01:51:40.587306+0000","last_fullsized":"2024-03-19T01:51:40.587306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:42:29.159512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:07.044189+0000","last_change":"2024-03-19T01:50:49.553780+0000","last_active":"2024-03-19T01:52:07.044189+0000","last_peered":"2024-03-19T01:52:07.044189+0000","last_clean":"2024-03-19T01:52:07.044189+0000","last_became_active":"2024-03-19T01:50:49.552795+0000","last_became_peered":"2024-03-19T01:50:49.552795+0000","last_unstale":"2024-03-19T01:52:07.044189+0000","last_undegraded":"2024-03-19T01:52:07.044189+0000","last_fullsized":"2024-03-19T01:52:07.044189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:01:37.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587351+0000","last_change":"2024-03-19T01:50:50.556943+0000","last_active":"2024-03-19T01:51:40.587351+0000","last_peered":"2024-03-19T01:51:40.587351+0000","last_clean":"2024-03-19T01:51:40.587351+0000","last_became_active":"2024-03-19T01:50:50.556646+0000","last_became_peered":"2024-03-19T01:50:50.556646+0000","last_unstale":"2024-03-19T01:51:40.587351+0000","last_undegraded":"2024-03-19T01:51:40.587351+0000","last_fullsized":"2024-03-19T01:51:40.587351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:59:28.712332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.852431+0000","last_change":"2024-03-19T01:50:49.549760+0000","last_active":"2024-03-19T01:51:40.852431+0000","last_peered":"2024-03-19T01:51:40.852431+0000","last_clean":"2024-03-19T01:51:40.852431+0000","last_became_active":"2024-03-19T01:50:49.549218+0000","last_became_peered":"2024-03-19T01:50:49.549218+0000","last_unstale":"2024-03-19T01:51:40.852431+0000","last_undegraded":"2024-03-19T01:51:40.852431+0000","last_fullsized":"2024-03-19T01:51:40.852431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:30:00.802902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584830+0000","last_change":"2024-03-19T01:49:36.469079+0000","last_active":"2024-03-19T01:51:40.584830+0000","last_peered":"2024-03-19T01:51:40.584830+0000","last_clean":"2024-03-19T01:51:40.584830+0000","last_became_active":"2024-03-19T01:49:36.468624+0000","last_became_peered":"2024-03-19T01:49:36.468624+0000","last_unstale":"2024-03-19T01:51:40.584830+0000","last_undegraded":"2024-03-19T01:51:40.584830+0000","last_fullsized":"2024-03-19T01:51:40.584830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:46:07.695833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:46:07.695833+0000","last_clean_scrub_stamp":"2024-03-19T01:46:07.695833+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:06:31.554853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.704775+0000","last_change":"2024-03-19T01:50:50.550851+0000","last_active":"2024-03-19T01:51:40.704775+0000","last_peered":"2024-03-19T01:51:40.704775+0000","last_clean":"2024-03-19T01:51:40.704775+0000","last_became_active":"2024-03-19T01:50:50.550581+0000","last_became_peered":"2024-03-19T01:50:50.550581+0000","last_unstale":"2024-03-19T01:51:40.704775+0000","last_undegraded":"2024-03-19T01:51:40.704775+0000","last_fullsized":"2024-03-19T01:51:40.704775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:24:24.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584919+0000","last_change":"2024-03-19T01:50:49.554043+0000","last_active":"2024-03-19T01:51:40.584919+0000","last_peered":"2024-03-19T01:51:40.584919+0000","last_clean":"2024-03-19T01:51:40.584919+0000","last_became_active":"2024-03-19T01:50:49.553597+0000","last_became_peered":"2024-03-19T01:50:49.553597+0000","last_unstale":"2024-03-19T01:51:40.584919+0000","last_undegraded":"2024-03-19T01:51:40.584919+0000","last_fullsized":"2024-03-19T01:51:40.584919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:41:40.545382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.586219+0000","last_change":"2024-03-19T01:50:49.553740+0000","last_active":"2024-03-19T01:51:40.586219+0000","last_peered":"2024-03-19T01:51:40.586219+0000","last_clean":"2024-03-19T01:51:40.586219+0000","last_became_active":"2024-03-19T01:50:49.553230+0000","last_became_peered":"2024-03-19T01:50:49.553230+0000","last_unstale":"2024-03-19T01:51:40.586219+0000","last_undegraded":"2024-03-19T01:51:40.586219+0000","last_fullsized":"2024-03-19T01:51:40.586219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:46:36.633880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705069+0000","last_change":"2024-03-19T01:50:50.554478+0000","last_active":"2024-03-19T01:51:40.705069+0000","last_peered":"2024-03-19T01:51:40.705069+0000","last_clean":"2024-03-19T01:51:40.705069+0000","last_became_active":"2024-03-19T01:50:50.551170+0000","last_became_peered":"2024-03-19T01:50:50.551170+0000","last_unstale":"2024-03-19T01:51:40.705069+0000","last_undegraded":"2024-03-19T01:51:40.705069+0000","last_fullsized":"2024-03-19T01:51:40.705069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:08:12.382239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588354+0000","last_change":"2024-03-19T01:50:49.554018+0000","last_active":"2024-03-19T01:51:40.588354+0000","last_peered":"2024-03-19T01:51:40.588354+0000","last_clean":"2024-03-19T01:51:40.588354+0000","last_became_active":"2024-03-19T01:50:49.553702+0000","last_became_peered":"2024-03-19T01:50:49.553702+0000","last_unstale":"2024-03-19T01:51:40.588354+0000","last_undegraded":"2024-03-19T01:51:40.588354+0000","last_fullsized":"2024-03-19T01:51:40.588354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:55.051901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588429+0000","last_change":"2024-03-19T01:50:50.565955+0000","last_active":"2024-03-19T01:51:40.588429+0000","last_peered":"2024-03-19T01:51:40.588429+0000","last_clean":"2024-03-19T01:51:40.588429+0000","last_became_active":"2024-03-19T01:50:50.565643+0000","last_became_peered":"2024-03-19T01:50:50.565643+0000","last_unstale":"2024-03-19T01:51:40.588429+0000","last_undegraded":"2024-03-19T01:51:40.588429+0000","last_fullsized":"2024-03-19T01:51:40.588429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:37:42.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.852765+0000","last_change":"2024-03-19T01:50:49.548298+0000","last_active":"2024-03-19T01:51:40.852765+0000","last_peered":"2024-03-19T01:51:40.852765+0000","last_clean":"2024-03-19T01:51:40.852765+0000","last_became_active":"2024-03-19T01:50:49.547941+0000","last_became_peered":"2024-03-19T01:50:49.547941+0000","last_unstale":"2024-03-19T01:51:40.852765+0000","last_undegraded":"2024-03-19T01:51:40.852765+0000","last_fullsized":"2024-03-19T01:51:40.852765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:23:59.221152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584731+0000","last_change":"2024-03-19T01:50:50.563923+0000","last_active":"2024-03-19T01:51:40.584731+0000","last_peered":"2024-03-19T01:51:40.584731+0000","last_clean":"2024-03-19T01:51:40.584731+0000","last_became_active":"2024-03-19T01:50:50.563295+0000","last_became_peered":"2024-03-19T01:50:50.563295+0000","last_unstale":"2024-03-19T01:51:40.584731+0000","last_undegraded":"2024-03-19T01:51:40.584731+0000","last_fullsized":"2024-03-19T01:51:40.584731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:09:47.089720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590012+0000","last_change":"2024-03-19T01:50:49.548014+0000","last_active":"2024-03-19T01:51:40.590012+0000","last_peered":"2024-03-19T01:51:40.590012+0000","last_clean":"2024-03-19T01:51:40.590012+0000","last_became_active":"2024-03-19T01:50:49.547588+0000","last_became_peered":"2024-03-19T01:50:49.547588+0000","last_unstale":"2024-03-19T01:51:40.590012+0000","last_undegraded":"2024-03-19T01:51:40.590012+0000","last_fullsized":"2024-03-19T01:51:40.590012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:08.258554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585622+0000","last_change":"2024-03-19T01:50:50.562437+0000","last_active":"2024-03-19T01:51:40.585622+0000","last_peered":"2024-03-19T01:51:40.585622+0000","last_clean":"2024-03-19T01:51:40.585622+0000","last_became_active":"2024-03-19T01:50:50.562113+0000","last_became_peered":"2024-03-19T01:50:50.562113+0000","last_unstale":"2024-03-19T01:51:40.585622+0000","last_undegraded":"2024-03-19T01:51:40.585622+0000","last_fullsized":"2024-03-19T01:51:40.585622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:19:31.932037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587239+0000","last_change":"2024-03-19T01:50:49.547190+0000","last_active":"2024-03-19T01:51:40.587239+0000","last_peered":"2024-03-19T01:51:40.587239+0000","last_clean":"2024-03-19T01:51:40.587239+0000","last_became_active":"2024-03-19T01:50:49.546698+0000","last_became_peered":"2024-03-19T01:50:49.546698+0000","last_unstale":"2024-03-19T01:51:40.587239+0000","last_undegraded":"2024-03-19T01:51:40.587239+0000","last_fullsized":"2024-03-19T01:51:40.587239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:09.557035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587925+0000","last_change":"2024-03-19T01:50:50.571159+0000","last_active":"2024-03-19T01:51:40.587925+0000","last_peered":"2024-03-19T01:51:40.587925+0000","last_clean":"2024-03-19T01:51:40.587925+0000","last_became_active":"2024-03-19T01:50:50.570312+0000","last_became_peered":"2024-03-19T01:50:50.570312+0000","last_unstale":"2024-03-19T01:51:40.587925+0000","last_undegraded":"2024-03-19T01:51:40.587925+0000","last_fullsized":"2024-03-19T01:51:40.587925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:23:16.820114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590785+0000","last_change":"2024-03-19T01:50:49.550557+0000","last_active":"2024-03-19T01:51:40.590785+0000","last_peered":"2024-03-19T01:51:40.590785+0000","last_clean":"2024-03-19T01:51:40.590785+0000","last_became_active":"2024-03-19T01:50:49.549789+0000","last_became_peered":"2024-03-19T01:50:49.549789+0000","last_unstale":"2024-03-19T01:51:40.590785+0000","last_undegraded":"2024-03-19T01:51:40.590785+0000","last_fullsized":"2024-03-19T01:51:40.590785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:06:14.292689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587678+0000","last_change":"2024-03-19T01:50:50.560116+0000","last_active":"2024-03-19T01:51:40.587678+0000","last_peered":"2024-03-19T01:51:40.587678+0000","last_clean":"2024-03-19T01:51:40.587678+0000","last_became_active":"2024-03-19T01:50:50.559809+0000","last_became_peered":"2024-03-19T01:50:50.559809+0000","last_unstale":"2024-03-19T01:51:40.587678+0000","last_undegraded":"2024-03-19T01:51:40.587678+0000","last_fullsized":"2024-03-19T01:51:40.587678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:26.854262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585620+0000","last_change":"2024-03-19T01:50:49.556100+0000","last_active":"2024-03-19T01:51:40.585620+0000","last_peered":"2024-03-19T01:51:40.585620+0000","last_clean":"2024-03-19T01:51:40.585620+0000","last_became_active":"2024-03-19T01:50:49.555342+0000","last_became_peered":"2024-03-19T01:50:49.555342+0000","last_unstale":"2024-03-19T01:51:40.585620+0000","last_undegraded":"2024-03-19T01:51:40.585620+0000","last_fullsized":"2024-03-19T01:51:40.585620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:50:15.712384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584944+0000","last_change":"2024-03-19T01:50:50.552792+0000","last_active":"2024-03-19T01:51:40.584944+0000","last_peered":"2024-03-19T01:51:40.584944+0000","last_clean":"2024-03-19T01:51:40.584944+0000","last_became_active":"2024-03-19T01:50:50.552438+0000","last_became_peered":"2024-03-19T01:50:50.552438+0000","last_unstale":"2024-03-19T01:51:40.584944+0000","last_undegraded":"2024-03-19T01:51:40.584944+0000","last_fullsized":"2024-03-19T01:51:40.584944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:42:10.436248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:06.043611+0000","last_change":"2024-03-19T01:50:49.546948+0000","last_active":"2024-03-19T01:52:06.043611+0000","last_peered":"2024-03-19T01:52:06.043611+0000","last_clean":"2024-03-19T01:52:06.043611+0000","last_became_active":"2024-03-19T01:50:49.546610+0000","last_became_peered":"2024-03-19T01:50:49.546610+0000","last_unstale":"2024-03-19T01:52:06.043611+0000","last_undegraded":"2024-03-19T01:52:06.043611+0000","last_fullsized":"2024-03-19T01:52:06.043611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:06:09.973827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585234+0000","last_change":"2024-03-19T01:50:50.552703+0000","last_active":"2024-03-19T01:51:40.585234+0000","last_peered":"2024-03-19T01:51:40.585234+0000","last_clean":"2024-03-19T01:51:40.585234+0000","last_became_active":"2024-03-19T01:50:50.552337+0000","last_became_peered":"2024-03-19T01:50:50.552337+0000","last_unstale":"2024-03-19T01:51:40.585234+0000","last_undegraded":"2024-03-19T01:51:40.585234+0000","last_fullsized":"2024-03-19T01:51:40.585234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:27.912513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587171+0000","last_change":"2024-03-19T01:50:49.554531+0000","last_active":"2024-03-19T01:51:40.587171+0000","last_peered":"2024-03-19T01:51:40.587171+0000","last_clean":"2024-03-19T01:51:40.587171+0000","last_became_active":"2024-03-19T01:50:49.554196+0000","last_became_peered":"2024-03-19T01:50:49.554196+0000","last_unstale":"2024-03-19T01:51:40.587171+0000","last_undegraded":"2024-03-19T01:51:40.587171+0000","last_fullsized":"2024-03-19T01:51:40.587171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:04.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.593555+0000","last_change":"2024-03-19T01:51:40.593555+0000","last_active":"2024-03-19T01:51:40.593555+0000","last_peered":"2024-03-19T01:51:40.593555+0000","last_clean":"2024-03-19T01:51:40.593555+0000","last_became_active":"2024-03-19T01:51:40.589104+0000","last_became_peered":"2024-03-19T01:51:40.589104+0000","last_unstale":"2024-03-19T01:51:40.593555+0000","last_undegraded":"2024-03-19T01:51:40.593555+0000","last_fullsized":"2024-03-19T01:51:40.593555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:25:44.319156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705010+0000","last_change":"2024-03-19T01:50:49.553203+0000","last_active":"2024-03-19T01:51:40.705010+0000","last_peered":"2024-03-19T01:51:40.705010+0000","last_clean":"2024-03-19T01:51:40.705010+0000","last_became_active":"2024-03-19T01:50:49.552243+0000","last_became_peered":"2024-03-19T01:50:49.552243+0000","last_unstale":"2024-03-19T01:51:40.705010+0000","last_undegraded":"2024-03-19T01:51:40.705010+0000","last_fullsized":"2024-03-19T01:51:40.705010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:16.313075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585207+0000","last_change":"2024-03-19T01:50:50.567317+0000","last_active":"2024-03-19T01:51:40.585207+0000","last_peered":"2024-03-19T01:51:40.585207+0000","last_clean":"2024-03-19T01:51:40.585207+0000","last_became_active":"2024-03-19T01:50:50.566744+0000","last_became_peered":"2024-03-19T01:50:50.566744+0000","last_unstale":"2024-03-19T01:51:40.585207+0000","last_undegraded":"2024-03-19T01:51:40.585207+0000","last_fullsized":"2024-03-19T01:51:40.585207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:01:34.239148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.592726+0000","last_change":"2024-03-19T01:51:40.592726+0000","last_active":"2024-03-19T01:51:40.592726+0000","last_peered":"2024-03-19T01:51:40.592726+0000","last_clean":"2024-03-19T01:51:40.592726+0000","last_became_active":"2024-03-19T01:51:40.592375+0000","last_became_peered":"2024-03-19T01:51:40.592375+0000","last_unstale":"2024-03-19T01:51:40.592726+0000","last_undegraded":"2024-03-19T01:51:40.592726+0000","last_fullsized":"2024-03-19T01:51:40.592726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:05:06.193052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590516+0000","last_change":"2024-03-19T01:50:50.564337+0000","last_active":"2024-03-19T01:51:40.590516+0000","last_peered":"2024-03-19T01:51:40.590516+0000","last_clean":"2024-03-19T01:51:40.590516+0000","last_became_active":"2024-03-19T01:50:50.564002+0000","last_became_peered":"2024-03-19T01:50:50.564002+0000","last_unstale":"2024-03-19T01:51:40.590516+0000","last_undegraded":"2024-03-19T01:51:40.590516+0000","last_fullsized":"2024-03-19T01:51:40.590516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:19:51.072829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590694+0000","last_change":"2024-03-19T01:50:50.560962+0000","last_active":"2024-03-19T01:51:40.590694+0000","last_peered":"2024-03-19T01:51:40.590694+0000","last_clean":"2024-03-19T01:51:40.590694+0000","last_became_active":"2024-03-19T01:50:50.560427+0000","last_became_peered":"2024-03-19T01:50:50.560427+0000","last_unstale":"2024-03-19T01:51:40.590694+0000","last_undegraded":"2024-03-19T01:51:40.590694+0000","last_fullsized":"2024-03-19T01:51:40.590694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:54.472200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584331+0000","last_change":"2024-03-19T01:50:50.557806+0000","last_active":"2024-03-19T01:51:40.584331+0000","last_peered":"2024-03-19T01:51:40.584331+0000","last_clean":"2024-03-19T01:51:40.584331+0000","last_became_active":"2024-03-19T01:50:50.557494+0000","last_became_peered":"2024-03-19T01:50:50.557494+0000","last_unstale":"2024-03-19T01:51:40.584331+0000","last_undegraded":"2024-03-19T01:51:40.584331+0000","last_fullsized":"2024-03-19T01:51:40.584331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:35:53.787047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705161+0000","last_change":"2024-03-19T01:50:50.554414+0000","last_active":"2024-03-19T01:51:40.705161+0000","last_peered":"2024-03-19T01:51:40.705161+0000","last_clean":"2024-03-19T01:51:40.705161+0000","last_became_active":"2024-03-19T01:50:50.554010+0000","last_became_peered":"2024-03-19T01:50:50.554010+0000","last_unstale":"2024-03-19T01:51:40.705161+0000","last_undegraded":"2024-03-19T01:51:40.705161+0000","last_fullsized":"2024-03-19T01:51:40.705161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:02:54.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588350+0000","last_change":"2024-03-19T01:50:50.567759+0000","last_active":"2024-03-19T01:51:40.588350+0000","last_peered":"2024-03-19T01:51:40.588350+0000","last_clean":"2024-03-19T01:51:40.588350+0000","last_became_active":"2024-03-19T01:50:50.566529+0000","last_became_peered":"2024-03-19T01:50:50.566529+0000","last_unstale":"2024-03-19T01:51:40.588350+0000","last_undegraded":"2024-03-19T01:51:40.588350+0000","last_fullsized":"2024-03-19T01:51:40.588350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:25:05.002161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590896+0000","last_change":"2024-03-19T01:50:50.563915+0000","last_active":"2024-03-19T01:51:40.590896+0000","last_peered":"2024-03-19T01:51:40.590896+0000","last_clean":"2024-03-19T01:51:40.590896+0000","last_became_active":"2024-03-19T01:50:50.563596+0000","last_became_peered":"2024-03-19T01:50:50.563596+0000","last_unstale":"2024-03-19T01:51:40.590896+0000","last_undegraded":"2024-03-19T01:51:40.590896+0000","last_fullsized":"2024-03-19T01:51:40.590896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:12.832402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588330+0000","last_change":"2024-03-19T01:50:50.556507+0000","last_active":"2024-03-19T01:51:40.588330+0000","last_peered":"2024-03-19T01:51:40.588330+0000","last_clean":"2024-03-19T01:51:40.588330+0000","last_became_active":"2024-03-19T01:50:50.556197+0000","last_became_peered":"2024-03-19T01:50:50.556197+0000","last_unstale":"2024-03-19T01:51:40.588330+0000","last_undegraded":"2024-03-19T01:51:40.588330+0000","last_fullsized":"2024-03-19T01:51:40.588330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:25:26.919274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587195+0000","last_change":"2024-03-19T01:50:50.554403+0000","last_active":"2024-03-19T01:51:40.587195+0000","last_peered":"2024-03-19T01:51:40.587195+0000","last_clean":"2024-03-19T01:51:40.587195+0000","last_became_active":"2024-03-19T01:50:50.553985+0000","last_became_peered":"2024-03-19T01:50:50.553985+0000","last_unstale":"2024-03-19T01:51:40.587195+0000","last_undegraded":"2024-03-19T01:51:40.587195+0000","last_fullsized":"2024-03-19T01:51:40.587195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:50.923915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587337+0000","last_change":"2024-03-19T01:50:50.554391+0000","last_active":"2024-03-19T01:51:40.587337+0000","last_peered":"2024-03-19T01:51:40.587337+0000","last_clean":"2024-03-19T01:51:40.587337+0000","last_became_active":"2024-03-19T01:50:50.552987+0000","last_became_peered":"2024-03-19T01:50:50.552987+0000","last_unstale":"2024-03-19T01:51:40.587337+0000","last_undegraded":"2024-03-19T01:51:40.587337+0000","last_fullsized":"2024-03-19T01:51:40.587337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:07.107613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590286+0000","last_change":"2024-03-19T01:50:50.556172+0000","last_active":"2024-03-19T01:51:40.590286+0000","last_peered":"2024-03-19T01:51:40.590286+0000","last_clean":"2024-03-19T01:51:40.590286+0000","last_became_active":"2024-03-19T01:50:50.555468+0000","last_became_peered":"2024-03-19T01:50:50.555468+0000","last_unstale":"2024-03-19T01:51:40.590286+0000","last_undegraded":"2024-03-19T01:51:40.590286+0000","last_fullsized":"2024-03-19T01:51:40.590286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:28:03.005962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705205+0000","last_change":"2024-03-19T01:50:50.550915+0000","last_active":"2024-03-19T01:51:40.705205+0000","last_peered":"2024-03-19T01:51:40.705205+0000","last_clean":"2024-03-19T01:51:40.705205+0000","last_became_active":"2024-03-19T01:50:50.550535+0000","last_became_peered":"2024-03-19T01:50:50.550535+0000","last_unstale":"2024-03-19T01:51:40.705205+0000","last_undegraded":"2024-03-19T01:51:40.705205+0000","last_fullsized":"2024-03-19T01:51:40.705205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:05:21.959483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588780+0000","last_change":"2024-03-19T01:50:50.567475+0000","last_active":"2024-03-19T01:51:40.588780+0000","last_peered":"2024-03-19T01:51:40.588780+0000","last_clean":"2024-03-19T01:51:40.588780+0000","last_became_active":"2024-03-19T01:50:50.567103+0000","last_became_peered":"2024-03-19T01:50:50.567103+0000","last_unstale":"2024-03-19T01:51:40.588780+0000","last_undegraded":"2024-03-19T01:51:40.588780+0000","last_fullsized":"2024-03-19T01:51:40.588780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:41.866610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244932,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66278,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":758,"num_read_kb":453,"num_write":236,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":198834,"data_compressed":5493,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":457449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645347,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1107293,"data_compressed":37525,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":456846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":456525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1049625,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":516824,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":457785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":517367,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":456154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1049303,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61217,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61760,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61201,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:52:11.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-19T01:52:11.611 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:11.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:11 smithi092 ceph-mon[62574]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:52:11.926 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:11 smithi092 ceph-mon[62574]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-19T01:52:11.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:11 smithi159 ceph-mon[67110]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:52:11.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:11 smithi159 ceph-mon[67110]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-19T01:52:12.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:11 smithi144 ceph-mon[67199]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:52:12.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:11 smithi144 ceph-mon[67199]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-19T01:52:12.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:12 smithi092 ceph-mon[62574]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:12 smithi159 ceph-mon[67110]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:12 smithi144 ceph-mon[67199]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:13.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 -- 172.21.15.92:0/2680185571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 msgr2=0x7fb3bc100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:13.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/2680185571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 0x7fb3bc100d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009990 tx=0x7fb3ac009ca0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 -- 172.21.15.92:0/2680185571 shutdown_connections 2024-03-19T01:52:13.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/2680185571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012c0 0x7fb3bc10f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/2680185571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc107380 0x7fb3bc107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/2680185571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 0x7fb3bc100d80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 -- 172.21.15.92:0/2680185571 >> 172.21.15.92:0/2680185571 conn(0x7fb3bc077810 msgr2=0x7fb3bc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 -- 172.21.15.92:0/2680185571 shutdown_connections 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 -- 172.21.15.92:0/2680185571 wait complete. 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.447+0000 7fb3c36c2700 1 Processor -- start 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c36c2700 1 -- start start 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c36c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 0x7fb3bc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c36c2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012c0 0x7fb3bc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c36c2700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc107380 0x7fb3bc198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c36c2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc112760 con 0x7fb3bc100900 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c36c2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1125e0 con 0x7fb3bc1012c0 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c36c2700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1128e0 con 0x7fb3bc107380 2024-03-19T01:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c145e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 0x7fb3bc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:13.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c0c5d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012c0 0x7fb3bc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:13.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c145e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 0x7fb3bc19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59704/0 (socket says 172.21.15.92:59704) 2024-03-19T01:52:13.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c145e700 1 -- 172.21.15.92:0/1350116214 learned_addr learned my addr 172.21.15.92:0/1350116214 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:13.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c1c5f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc107380 0x7fb3bc198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:13.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c1c5f700 1 -- 172.21.15.92:0/1350116214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012c0 msgr2=0x7fb3bc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:13.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c1c5f700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012c0 0x7fb3bc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c1c5f700 1 -- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 msgr2=0x7fb3bc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:13.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c1c5f700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 0x7fb3bc19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c1c5f700 1 -- 172.21.15.92:0/1350116214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc199160 con 0x7fb3bc107380 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c0c5d700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012c0 0x7fb3bc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.448+0000 7fb3c145e700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 0x7fb3bc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.449+0000 7fb3c1c5f700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc107380 0x7fb3bc198a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400ef90 tx=0x7fb3b400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.449+0000 7fb3b27fc700 1 -- 172.21.15.92:0/1350116214 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40092e0 con 0x7fb3bc107380 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.449+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc1993f0 con 0x7fb3bc107380 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.449+0000 7fb3b27fc700 1 -- 172.21.15.92:0/1350116214 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b4004bb0 con 0x7fb3bc107380 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.449+0000 7fb3b27fc700 1 -- 172.21.15.92:0/1350116214 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004280 con 0x7fb3bc107380 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.449+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc1a5710 con 0x7fb3bc107380 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.450+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc10a8e0 con 0x7fb3bc107380 2024-03-19T01:52:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.451+0000 7fb3b27fc700 1 -- 172.21.15.92:0/1350116214 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb3b401b030 con 0x7fb3bc107380 2024-03-19T01:52:13.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.452+0000 7fb3b27fc700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2890 0x7fb3a80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:13.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.452+0000 7fb3c145e700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2890 0x7fb3a80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:13.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.452+0000 7fb3b27fc700 1 -- 172.21.15.92:0/1350116214 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb3b40ce7b0 con 0x7fb3bc107380 2024-03-19T01:52:13.453 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.452+0000 7fb3c145e700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2890 0x7fb3a80a4d50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc199b40 tx=0x7fb3b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:13.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.458+0000 7fb3b27fc700 1 -- 172.21.15.92:0/1350116214 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb3b4022020 con 0x7fb3bc107380 2024-03-19T01:52:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.707+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3bc199f30 con 0x7fb3a80a2890 2024-03-19T01:52:13.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.718+0000 7fb3b27fc700 1 -- 172.21.15.92:0/1350116214 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259146 (secure 0 0 0) 0x7fb3bc199f30 con 0x7fb3a80a2890 2024-03-19T01:52:13.722 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2890 msgr2=0x7fb3a80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2890 0x7fb3a80a4d50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc199b40 tx=0x7fb3b8009260 comp rx=0 tx=0).stop 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc107380 msgr2=0x7fb3bc198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc107380 0x7fb3bc198a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400ef90 tx=0x7fb3b400c3a0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 shutdown_connections 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2890 0x7fb3a80a4d50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc107380 0x7fb3bc198a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100900 0x7fb3bc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 --2- 172.21.15.92:0/1350116214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012c0 0x7fb3bc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 >> 172.21.15.92:0/1350116214 conn(0x7fb3bc077810 msgr2=0x7fb3bc1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 shutdown_connections 2024-03-19T01:52:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:13.721+0000 7fb3c36c2700 1 -- 172.21.15.92:0/1350116214 wait complete. 2024-03-19T01:52:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-03-19T01:52:13.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:13 smithi159 ceph-mon[67110]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-19T01:52:14.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:13 smithi092 ceph-mon[62574]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-19T01:52:14.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:13 smithi144 ceph-mon[67199]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-19T01:52:14.471 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-03-19T01:52:13.476972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657270,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":931,"num_read_kb":584,"num_write":426,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1181503,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":687,"ondisk_log_size":687,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352548,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589276,"statfs":{"total":1151940427776,"available":1151579418624,"internally_reserved":0,"allocated":11980800,"data_stored":7438845,"data_compressed":112575,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19140,"internal_metadata":348894524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":72,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003290"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.496177+0000","last_change":"2024-03-19T01:50:50.561781+0000","last_active":"2024-03-19T01:52:09.496177+0000","last_peered":"2024-03-19T01:52:09.496177+0000","last_clean":"2024-03-19T01:52:09.496177+0000","last_became_active":"2024-03-19T01:50:50.561402+0000","last_became_peered":"2024-03-19T01:50:50.561402+0000","last_unstale":"2024-03-19T01:52:09.496177+0000","last_undegraded":"2024-03-19T01:52:09.496177+0000","last_fullsized":"2024-03-19T01:52:09.496177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:28:07.257394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584639+0000","last_change":"2024-03-19T01:50:50.558625+0000","last_active":"2024-03-19T01:51:40.584639+0000","last_peered":"2024-03-19T01:51:40.584639+0000","last_clean":"2024-03-19T01:51:40.584639+0000","last_became_active":"2024-03-19T01:50:50.557349+0000","last_became_peered":"2024-03-19T01:50:50.557349+0000","last_unstale":"2024-03-19T01:51:40.584639+0000","last_undegraded":"2024-03-19T01:51:40.584639+0000","last_fullsized":"2024-03-19T01:51:40.584639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:05.791529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584544+0000","last_change":"2024-03-19T01:50:50.554103+0000","last_active":"2024-03-19T01:51:40.584544+0000","last_peered":"2024-03-19T01:51:40.584544+0000","last_clean":"2024-03-19T01:51:40.584544+0000","last_became_active":"2024-03-19T01:50:50.553734+0000","last_became_peered":"2024-03-19T01:50:50.553734+0000","last_unstale":"2024-03-19T01:51:40.584544+0000","last_undegraded":"2024-03-19T01:51:40.584544+0000","last_fullsized":"2024-03-19T01:51:40.584544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:18:02.768774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.486766+0000","last_change":"2024-03-19T01:50:50.571980+0000","last_active":"2024-03-19T01:52:09.486766+0000","last_peered":"2024-03-19T01:52:09.486766+0000","last_clean":"2024-03-19T01:52:09.486766+0000","last_became_active":"2024-03-19T01:50:50.571679+0000","last_became_peered":"2024-03-19T01:50:50.571679+0000","last_unstale":"2024-03-19T01:52:09.486766+0000","last_undegraded":"2024-03-19T01:52:09.486766+0000","last_fullsized":"2024-03-19T01:52:09.486766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:30:08.946509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.474171+0000","last_change":"2024-03-19T01:50:50.563314+0000","last_active":"2024-03-19T01:52:09.474171+0000","last_peered":"2024-03-19T01:52:09.474171+0000","last_clean":"2024-03-19T01:52:09.474171+0000","last_became_active":"2024-03-19T01:50:50.562953+0000","last_became_peered":"2024-03-19T01:50:50.562953+0000","last_unstale":"2024-03-19T01:52:09.474171+0000","last_undegraded":"2024-03-19T01:52:09.474171+0000","last_fullsized":"2024-03-19T01:52:09.474171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:38.139670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.481621+0000","last_change":"2024-03-19T01:50:50.555305+0000","last_active":"2024-03-19T01:52:09.481621+0000","last_peered":"2024-03-19T01:52:09.481621+0000","last_clean":"2024-03-19T01:52:09.481621+0000","last_became_active":"2024-03-19T01:50:50.554930+0000","last_became_peered":"2024-03-19T01:50:50.554930+0000","last_unstale":"2024-03-19T01:52:09.481621+0000","last_undegraded":"2024-03-19T01:52:09.481621+0000","last_fullsized":"2024-03-19T01:52:09.481621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:05:03.418572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.480616+0000","last_change":"2024-03-19T01:50:50.557381+0000","last_active":"2024-03-19T01:52:09.480616+0000","last_peered":"2024-03-19T01:52:09.480616+0000","last_clean":"2024-03-19T01:52:09.480616+0000","last_became_active":"2024-03-19T01:50:50.556804+0000","last_became_peered":"2024-03-19T01:50:50.556804+0000","last_unstale":"2024-03-19T01:52:09.480616+0000","last_undegraded":"2024-03-19T01:52:09.480616+0000","last_fullsized":"2024-03-19T01:52:09.480616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:54:00.676606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.481821+0000","last_change":"2024-03-19T01:51:40.591005+0000","last_active":"2024-03-19T01:52:09.481821+0000","last_peered":"2024-03-19T01:52:09.481821+0000","last_clean":"2024-03-19T01:52:09.481821+0000","last_became_active":"2024-03-19T01:51:40.590558+0000","last_became_peered":"2024-03-19T01:51:40.590558+0000","last_unstale":"2024-03-19T01:52:09.481821+0000","last_undegraded":"2024-03-19T01:52:09.481821+0000","last_fullsized":"2024-03-19T01:52:09.481821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:19:45.880448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590704+0000","last_change":"2024-03-19T01:50:50.562970+0000","last_active":"2024-03-19T01:51:40.590704+0000","last_peered":"2024-03-19T01:51:40.590704+0000","last_clean":"2024-03-19T01:51:40.590704+0000","last_became_active":"2024-03-19T01:50:50.562372+0000","last_became_peered":"2024-03-19T01:50:50.562372+0000","last_unstale":"2024-03-19T01:51:40.590704+0000","last_undegraded":"2024-03-19T01:51:40.590704+0000","last_fullsized":"2024-03-19T01:51:40.590704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:57.861083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.488651+0000","last_change":"2024-03-19T01:50:50.559369+0000","last_active":"2024-03-19T01:52:09.488651+0000","last_peered":"2024-03-19T01:52:09.488651+0000","last_clean":"2024-03-19T01:52:09.488651+0000","last_became_active":"2024-03-19T01:50:50.559062+0000","last_became_peered":"2024-03-19T01:50:50.559062+0000","last_unstale":"2024-03-19T01:52:09.488651+0000","last_undegraded":"2024-03-19T01:52:09.488651+0000","last_fullsized":"2024-03-19T01:52:09.488651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:08:22.977566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.492297+0000","last_change":"2024-03-19T01:50:50.567000+0000","last_active":"2024-03-19T01:52:09.492297+0000","last_peered":"2024-03-19T01:52:09.492297+0000","last_clean":"2024-03-19T01:52:09.492297+0000","last_became_active":"2024-03-19T01:50:50.566613+0000","last_became_peered":"2024-03-19T01:50:50.566613+0000","last_unstale":"2024-03-19T01:52:09.492297+0000","last_undegraded":"2024-03-19T01:52:09.492297+0000","last_fullsized":"2024-03-19T01:52:09.492297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:40:39.132559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.478992+0000","last_change":"2024-03-19T01:50:50.564354+0000","last_active":"2024-03-19T01:52:09.478992+0000","last_peered":"2024-03-19T01:52:09.478992+0000","last_clean":"2024-03-19T01:52:09.478992+0000","last_became_active":"2024-03-19T01:50:50.563997+0000","last_became_peered":"2024-03-19T01:50:50.563997+0000","last_unstale":"2024-03-19T01:52:09.478992+0000","last_undegraded":"2024-03-19T01:52:09.478992+0000","last_fullsized":"2024-03-19T01:52:09.478992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:27:50.266867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705190+0000","last_change":"2024-03-19T01:50:50.539164+0000","last_active":"2024-03-19T01:51:40.705190+0000","last_peered":"2024-03-19T01:51:40.705190+0000","last_clean":"2024-03-19T01:51:40.705190+0000","last_became_active":"2024-03-19T01:50:50.538792+0000","last_became_peered":"2024-03-19T01:50:50.538792+0000","last_unstale":"2024-03-19T01:51:40.705190+0000","last_undegraded":"2024-03-19T01:51:40.705190+0000","last_fullsized":"2024-03-19T01:51:40.705190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:07:56.700994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584286+0000","last_change":"2024-03-19T01:50:50.567590+0000","last_active":"2024-03-19T01:51:40.584286+0000","last_peered":"2024-03-19T01:51:40.584286+0000","last_clean":"2024-03-19T01:51:40.584286+0000","last_became_active":"2024-03-19T01:50:50.567225+0000","last_became_peered":"2024-03-19T01:50:50.567225+0000","last_unstale":"2024-03-19T01:51:40.584286+0000","last_undegraded":"2024-03-19T01:51:40.584286+0000","last_fullsized":"2024-03-19T01:51:40.584286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:55.537567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.592111+0000","last_change":"2024-03-19T01:51:40.592111+0000","last_active":"2024-03-19T01:51:40.592111+0000","last_peered":"2024-03-19T01:51:40.592111+0000","last_clean":"2024-03-19T01:51:40.592111+0000","last_became_active":"2024-03-19T01:51:40.591725+0000","last_became_peered":"2024-03-19T01:51:40.591725+0000","last_unstale":"2024-03-19T01:51:40.592111+0000","last_undegraded":"2024-03-19T01:51:40.592111+0000","last_fullsized":"2024-03-19T01:51:40.592111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:05:36.932918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.589971+0000","last_change":"2024-03-19T01:50:50.558616+0000","last_active":"2024-03-19T01:51:40.589971+0000","last_peered":"2024-03-19T01:51:40.589971+0000","last_clean":"2024-03-19T01:51:40.589971+0000","last_became_active":"2024-03-19T01:50:50.558173+0000","last_became_peered":"2024-03-19T01:50:50.558173+0000","last_unstale":"2024-03-19T01:51:40.589971+0000","last_undegraded":"2024-03-19T01:51:40.589971+0000","last_fullsized":"2024-03-19T01:51:40.589971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:04:47.174828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.496284+0000","last_change":"2024-03-19T01:50:50.563450+0000","last_active":"2024-03-19T01:52:09.496284+0000","last_peered":"2024-03-19T01:52:09.496284+0000","last_clean":"2024-03-19T01:52:09.496284+0000","last_became_active":"2024-03-19T01:50:50.562663+0000","last_became_peered":"2024-03-19T01:50:50.562663+0000","last_unstale":"2024-03-19T01:52:09.496284+0000","last_undegraded":"2024-03-19T01:52:09.496284+0000","last_fullsized":"2024-03-19T01:52:09.496284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:57:47.262774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.488562+0000","last_change":"2024-03-19T01:50:50.559848+0000","last_active":"2024-03-19T01:52:09.488562+0000","last_peered":"2024-03-19T01:52:09.488562+0000","last_clean":"2024-03-19T01:52:09.488562+0000","last_became_active":"2024-03-19T01:50:50.559515+0000","last_became_peered":"2024-03-19T01:50:50.559515+0000","last_unstale":"2024-03-19T01:52:09.488562+0000","last_undegraded":"2024-03-19T01:52:09.488562+0000","last_fullsized":"2024-03-19T01:52:09.488562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:16:15.262613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.482061+0000","last_change":"2024-03-19T01:50:50.559994+0000","last_active":"2024-03-19T01:52:09.482061+0000","last_peered":"2024-03-19T01:52:09.482061+0000","last_clean":"2024-03-19T01:52:09.482061+0000","last_became_active":"2024-03-19T01:50:50.558759+0000","last_became_peered":"2024-03-19T01:50:50.558759+0000","last_unstale":"2024-03-19T01:52:09.482061+0000","last_undegraded":"2024-03-19T01:52:09.482061+0000","last_fullsized":"2024-03-19T01:52:09.482061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:53.957096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:10.056385+0000","last_change":"2024-03-19T01:50:49.550130+0000","last_active":"2024-03-19T01:52:10.056385+0000","last_peered":"2024-03-19T01:52:10.056385+0000","last_clean":"2024-03-19T01:52:10.056385+0000","last_became_active":"2024-03-19T01:50:49.549374+0000","last_became_peered":"2024-03-19T01:50:49.549374+0000","last_unstale":"2024-03-19T01:52:10.056385+0000","last_undegraded":"2024-03-19T01:52:10.056385+0000","last_fullsized":"2024-03-19T01:52:10.056385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:14:43.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.493234+0000","last_change":"2024-03-19T01:50:50.570319+0000","last_active":"2024-03-19T01:52:09.493234+0000","last_peered":"2024-03-19T01:52:09.493234+0000","last_clean":"2024-03-19T01:52:09.493234+0000","last_became_active":"2024-03-19T01:50:50.569937+0000","last_became_peered":"2024-03-19T01:50:50.569937+0000","last_unstale":"2024-03-19T01:52:09.493234+0000","last_undegraded":"2024-03-19T01:52:09.493234+0000","last_fullsized":"2024-03-19T01:52:09.493234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:38:00.024688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588415+0000","last_change":"2024-03-19T01:50:49.551261+0000","last_active":"2024-03-19T01:51:40.588415+0000","last_peered":"2024-03-19T01:51:40.588415+0000","last_clean":"2024-03-19T01:51:40.588415+0000","last_became_active":"2024-03-19T01:50:49.550949+0000","last_became_peered":"2024-03-19T01:50:49.550949+0000","last_unstale":"2024-03-19T01:51:40.588415+0000","last_undegraded":"2024-03-19T01:51:40.588415+0000","last_fullsized":"2024-03-19T01:51:40.588415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:23:49.064155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584912+0000","last_change":"2024-03-19T01:50:50.554303+0000","last_active":"2024-03-19T01:51:40.584912+0000","last_peered":"2024-03-19T01:51:40.584912+0000","last_clean":"2024-03-19T01:51:40.584912+0000","last_became_active":"2024-03-19T01:50:50.553765+0000","last_became_peered":"2024-03-19T01:50:50.553765+0000","last_unstale":"2024-03-19T01:51:40.584912+0000","last_undegraded":"2024-03-19T01:51:40.584912+0000","last_fullsized":"2024-03-19T01:51:40.584912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:42.246266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.586323+0000","last_change":"2024-03-19T01:50:49.554688+0000","last_active":"2024-03-19T01:51:40.586323+0000","last_peered":"2024-03-19T01:51:40.586323+0000","last_clean":"2024-03-19T01:51:40.586323+0000","last_became_active":"2024-03-19T01:50:49.554289+0000","last_became_peered":"2024-03-19T01:50:49.554289+0000","last_unstale":"2024-03-19T01:51:40.586323+0000","last_undegraded":"2024-03-19T01:51:40.586323+0000","last_fullsized":"2024-03-19T01:51:40.586323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:07:13.316243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590266+0000","last_change":"2024-03-19T01:50:50.563305+0000","last_active":"2024-03-19T01:51:40.590266+0000","last_peered":"2024-03-19T01:51:40.590266+0000","last_clean":"2024-03-19T01:51:40.590266+0000","last_became_active":"2024-03-19T01:50:50.561819+0000","last_became_peered":"2024-03-19T01:50:50.561819+0000","last_unstale":"2024-03-19T01:51:40.590266+0000","last_undegraded":"2024-03-19T01:51:40.590266+0000","last_fullsized":"2024-03-19T01:51:40.590266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:11:56.175649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.599378+0000","last_change":"2024-03-19T01:51:40.599378+0000","last_active":"2024-03-19T01:51:40.599378+0000","last_peered":"2024-03-19T01:51:40.599378+0000","last_clean":"2024-03-19T01:51:40.599378+0000","last_became_active":"2024-03-19T01:51:40.592404+0000","last_became_peered":"2024-03-19T01:51:40.592404+0000","last_unstale":"2024-03-19T01:51:40.599378+0000","last_undegraded":"2024-03-19T01:51:40.599378+0000","last_fullsized":"2024-03-19T01:51:40.599378+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:03:37.485954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.495438+0000","last_change":"2024-03-19T01:50:50.560178+0000","last_active":"2024-03-19T01:52:09.495438+0000","last_peered":"2024-03-19T01:52:09.495438+0000","last_clean":"2024-03-19T01:52:09.495438+0000","last_became_active":"2024-03-19T01:50:50.559853+0000","last_became_peered":"2024-03-19T01:50:50.559853+0000","last_unstale":"2024-03-19T01:52:09.495438+0000","last_undegraded":"2024-03-19T01:52:09.495438+0000","last_fullsized":"2024-03-19T01:52:09.495438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:10:04.812207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:12.050473+0000","last_change":"2024-03-19T01:50:49.550194+0000","last_active":"2024-03-19T01:52:12.050473+0000","last_peered":"2024-03-19T01:52:12.050473+0000","last_clean":"2024-03-19T01:52:12.050473+0000","last_became_active":"2024-03-19T01:50:49.549801+0000","last_became_peered":"2024-03-19T01:50:49.549801+0000","last_unstale":"2024-03-19T01:52:12.050473+0000","last_undegraded":"2024-03-19T01:52:12.050473+0000","last_fullsized":"2024-03-19T01:52:12.050473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:11:43.110159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.470033+0000","last_change":"2024-03-19T01:50:50.560913+0000","last_active":"2024-03-19T01:52:09.470033+0000","last_peered":"2024-03-19T01:52:09.470033+0000","last_clean":"2024-03-19T01:52:09.470033+0000","last_became_active":"2024-03-19T01:50:50.559210+0000","last_became_peered":"2024-03-19T01:50:50.559210+0000","last_unstale":"2024-03-19T01:52:09.470033+0000","last_undegraded":"2024-03-19T01:52:09.470033+0000","last_fullsized":"2024-03-19T01:52:09.470033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:08:14.814374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.589771+0000","last_change":"2024-03-19T01:50:49.548778+0000","last_active":"2024-03-19T01:51:40.589771+0000","last_peered":"2024-03-19T01:51:40.589771+0000","last_clean":"2024-03-19T01:51:40.589771+0000","last_became_active":"2024-03-19T01:50:49.548414+0000","last_became_peered":"2024-03-19T01:50:49.548414+0000","last_unstale":"2024-03-19T01:51:40.589771+0000","last_undegraded":"2024-03-19T01:51:40.589771+0000","last_fullsized":"2024-03-19T01:51:40.589771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:03:37.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.492260+0000","last_change":"2024-03-19T01:50:50.559242+0000","last_active":"2024-03-19T01:52:09.492260+0000","last_peered":"2024-03-19T01:52:09.492260+0000","last_clean":"2024-03-19T01:52:09.492260+0000","last_became_active":"2024-03-19T01:50:50.558904+0000","last_became_peered":"2024-03-19T01:50:50.558904+0000","last_unstale":"2024-03-19T01:52:09.492260+0000","last_undegraded":"2024-03-19T01:52:09.492260+0000","last_fullsized":"2024-03-19T01:52:09.492260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:25:59.249180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.704875+0000","last_change":"2024-03-19T01:50:49.548331+0000","last_active":"2024-03-19T01:51:40.704875+0000","last_peered":"2024-03-19T01:51:40.704875+0000","last_clean":"2024-03-19T01:51:40.704875+0000","last_became_active":"2024-03-19T01:50:49.547993+0000","last_became_peered":"2024-03-19T01:50:49.547993+0000","last_unstale":"2024-03-19T01:51:40.704875+0000","last_undegraded":"2024-03-19T01:51:40.704875+0000","last_fullsized":"2024-03-19T01:51:40.704875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:30.059133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.485367+0000","last_change":"2024-03-19T01:50:50.553166+0000","last_active":"2024-03-19T01:52:09.485367+0000","last_peered":"2024-03-19T01:52:09.485367+0000","last_clean":"2024-03-19T01:52:09.485367+0000","last_became_active":"2024-03-19T01:50:50.552520+0000","last_became_peered":"2024-03-19T01:50:50.552520+0000","last_unstale":"2024-03-19T01:52:09.485367+0000","last_undegraded":"2024-03-19T01:52:09.485367+0000","last_fullsized":"2024-03-19T01:52:09.485367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:10:24.515191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587095+0000","last_change":"2024-03-19T01:50:49.548868+0000","last_active":"2024-03-19T01:51:40.587095+0000","last_peered":"2024-03-19T01:51:40.587095+0000","last_clean":"2024-03-19T01:51:40.587095+0000","last_became_active":"2024-03-19T01:50:49.548548+0000","last_became_peered":"2024-03-19T01:50:49.548548+0000","last_unstale":"2024-03-19T01:51:40.587095+0000","last_undegraded":"2024-03-19T01:51:40.587095+0000","last_fullsized":"2024-03-19T01:51:40.587095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:21:36.920067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.481162+0000","last_change":"2024-03-19T01:50:50.559870+0000","last_active":"2024-03-19T01:52:09.481162+0000","last_peered":"2024-03-19T01:52:09.481162+0000","last_clean":"2024-03-19T01:52:09.481162+0000","last_became_active":"2024-03-19T01:50:50.559530+0000","last_became_peered":"2024-03-19T01:50:50.559530+0000","last_unstale":"2024-03-19T01:52:09.481162+0000","last_undegraded":"2024-03-19T01:52:09.481162+0000","last_fullsized":"2024-03-19T01:52:09.481162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:39:48.635416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":180,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:13.050257+0000","last_change":"2024-03-19T01:50:49.554733+0000","last_active":"2024-03-19T01:52:13.050257+0000","last_peered":"2024-03-19T01:52:13.050257+0000","last_clean":"2024-03-19T01:52:13.050257+0000","last_became_active":"2024-03-19T01:50:49.554366+0000","last_became_peered":"2024-03-19T01:50:49.554366+0000","last_unstale":"2024-03-19T01:52:13.050257+0000","last_undegraded":"2024-03-19T01:52:13.050257+0000","last_fullsized":"2024-03-19T01:52:13.050257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:53:03.790255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590608+0000","last_change":"2024-03-19T01:50:50.561526+0000","last_active":"2024-03-19T01:51:40.590608+0000","last_peered":"2024-03-19T01:51:40.590608+0000","last_clean":"2024-03-19T01:51:40.590608+0000","last_became_active":"2024-03-19T01:50:50.561189+0000","last_became_peered":"2024-03-19T01:50:50.561189+0000","last_unstale":"2024-03-19T01:51:40.590608+0000","last_undegraded":"2024-03-19T01:51:40.590608+0000","last_fullsized":"2024-03-19T01:51:40.590608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:15:18.912920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585125+0000","last_change":"2024-03-19T01:50:49.553701+0000","last_active":"2024-03-19T01:51:40.585125+0000","last_peered":"2024-03-19T01:51:40.585125+0000","last_clean":"2024-03-19T01:51:40.585125+0000","last_became_active":"2024-03-19T01:50:49.553079+0000","last_became_peered":"2024-03-19T01:50:49.553079+0000","last_unstale":"2024-03-19T01:51:40.585125+0000","last_undegraded":"2024-03-19T01:51:40.585125+0000","last_fullsized":"2024-03-19T01:51:40.585125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:19.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.706526+0000","last_change":"2024-03-19T01:51:40.706526+0000","last_active":"2024-03-19T01:51:40.706526+0000","last_peered":"2024-03-19T01:51:40.706526+0000","last_clean":"2024-03-19T01:51:40.706526+0000","last_became_active":"2024-03-19T01:51:40.706109+0000","last_became_peered":"2024-03-19T01:51:40.706109+0000","last_unstale":"2024-03-19T01:51:40.706526+0000","last_undegraded":"2024-03-19T01:51:40.706526+0000","last_fullsized":"2024-03-19T01:51:40.706526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:14:17.713288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587354+0000","last_change":"2024-03-19T01:50:49.553573+0000","last_active":"2024-03-19T01:51:40.587354+0000","last_peered":"2024-03-19T01:51:40.587354+0000","last_clean":"2024-03-19T01:51:40.587354+0000","last_became_active":"2024-03-19T01:50:49.553209+0000","last_became_peered":"2024-03-19T01:50:49.553209+0000","last_unstale":"2024-03-19T01:51:40.587354+0000","last_undegraded":"2024-03-19T01:51:40.587354+0000","last_fullsized":"2024-03-19T01:51:40.587354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:47:43.235323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590313+0000","last_change":"2024-03-19T01:50:50.568133+0000","last_active":"2024-03-19T01:51:40.590313+0000","last_peered":"2024-03-19T01:51:40.590313+0000","last_clean":"2024-03-19T01:51:40.590313+0000","last_became_active":"2024-03-19T01:50:50.567783+0000","last_became_peered":"2024-03-19T01:50:50.567783+0000","last_unstale":"2024-03-19T01:51:40.590313+0000","last_undegraded":"2024-03-19T01:51:40.590313+0000","last_fullsized":"2024-03-19T01:51:40.590313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:16.077111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":215,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:13.050338+0000","last_change":"2024-03-19T01:50:49.546463+0000","last_active":"2024-03-19T01:52:13.050338+0000","last_peered":"2024-03-19T01:52:13.050338+0000","last_clean":"2024-03-19T01:52:13.050338+0000","last_became_active":"2024-03-19T01:50:49.546033+0000","last_became_peered":"2024-03-19T01:50:49.546033+0000","last_unstale":"2024-03-19T01:52:13.050338+0000","last_undegraded":"2024-03-19T01:52:13.050338+0000","last_fullsized":"2024-03-19T01:52:13.050338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:29:13.933301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59714,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":164,"num_read_kb":141,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.493602+0000","last_change":"2024-03-19T01:50:50.557576+0000","last_active":"2024-03-19T01:52:09.493602+0000","last_peered":"2024-03-19T01:52:09.493602+0000","last_clean":"2024-03-19T01:52:09.493602+0000","last_became_active":"2024-03-19T01:50:50.555024+0000","last_became_peered":"2024-03-19T01:50:50.555024+0000","last_unstale":"2024-03-19T01:52:09.493602+0000","last_undegraded":"2024-03-19T01:52:09.493602+0000","last_fullsized":"2024-03-19T01:52:09.493602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:17:17.252079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.608019+0000","last_change":"2024-03-19T01:51:40.608019+0000","last_active":"2024-03-19T01:51:40.608019+0000","last_peered":"2024-03-19T01:51:40.608019+0000","last_clean":"2024-03-19T01:51:40.608019+0000","last_became_active":"2024-03-19T01:51:40.592774+0000","last_became_peered":"2024-03-19T01:51:40.592774+0000","last_unstale":"2024-03-19T01:51:40.608019+0000","last_undegraded":"2024-03-19T01:51:40.608019+0000","last_fullsized":"2024-03-19T01:51:40.608019+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:36:47.204951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.704972+0000","last_change":"2024-03-19T01:50:50.554679+0000","last_active":"2024-03-19T01:51:40.704972+0000","last_peered":"2024-03-19T01:51:40.704972+0000","last_clean":"2024-03-19T01:51:40.704972+0000","last_became_active":"2024-03-19T01:50:50.554361+0000","last_became_peered":"2024-03-19T01:50:50.554361+0000","last_unstale":"2024-03-19T01:51:40.704972+0000","last_undegraded":"2024-03-19T01:51:40.704972+0000","last_fullsized":"2024-03-19T01:51:40.704972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:44:53.385907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587276+0000","last_change":"2024-03-19T01:50:49.547590+0000","last_active":"2024-03-19T01:51:40.587276+0000","last_peered":"2024-03-19T01:51:40.587276+0000","last_clean":"2024-03-19T01:51:40.587276+0000","last_became_active":"2024-03-19T01:50:49.547261+0000","last_became_peered":"2024-03-19T01:50:49.547261+0000","last_unstale":"2024-03-19T01:51:40.587276+0000","last_undegraded":"2024-03-19T01:51:40.587276+0000","last_fullsized":"2024-03-19T01:51:40.587276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:47.353161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.483929+0000","last_change":"2024-03-19T01:50:50.554830+0000","last_active":"2024-03-19T01:52:09.483929+0000","last_peered":"2024-03-19T01:52:09.483929+0000","last_clean":"2024-03-19T01:52:09.483929+0000","last_became_active":"2024-03-19T01:50:50.554495+0000","last_became_peered":"2024-03-19T01:50:50.554495+0000","last_unstale":"2024-03-19T01:52:09.483929+0000","last_undegraded":"2024-03-19T01:52:09.483929+0000","last_fullsized":"2024-03-19T01:52:09.483929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:44.098279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:12.049579+0000","last_change":"2024-03-19T01:50:49.550056+0000","last_active":"2024-03-19T01:52:12.049579+0000","last_peered":"2024-03-19T01:52:12.049579+0000","last_clean":"2024-03-19T01:52:12.049579+0000","last_became_active":"2024-03-19T01:50:49.549404+0000","last_became_peered":"2024-03-19T01:50:49.549404+0000","last_unstale":"2024-03-19T01:52:12.049579+0000","last_undegraded":"2024-03-19T01:52:12.049579+0000","last_fullsized":"2024-03-19T01:52:12.049579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:38:38.656100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":81,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.479854+0000","last_change":"2024-03-19T01:50:50.559443+0000","last_active":"2024-03-19T01:52:09.479854+0000","last_peered":"2024-03-19T01:52:09.479854+0000","last_clean":"2024-03-19T01:52:09.479854+0000","last_became_active":"2024-03-19T01:50:50.558992+0000","last_became_peered":"2024-03-19T01:50:50.558992+0000","last_unstale":"2024-03-19T01:52:09.479854+0000","last_undegraded":"2024-03-19T01:52:09.479854+0000","last_fullsized":"2024-03-19T01:52:09.479854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:06:52.144816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585516+0000","last_change":"2024-03-19T01:50:49.548423+0000","last_active":"2024-03-19T01:51:40.585516+0000","last_peered":"2024-03-19T01:51:40.585516+0000","last_clean":"2024-03-19T01:51:40.585516+0000","last_became_active":"2024-03-19T01:50:49.548071+0000","last_became_peered":"2024-03-19T01:50:49.548071+0000","last_unstale":"2024-03-19T01:51:40.585516+0000","last_undegraded":"2024-03-19T01:51:40.585516+0000","last_fullsized":"2024-03-19T01:51:40.585516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:06:00.789358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.490183+0000","last_change":"2024-03-19T01:50:50.558569+0000","last_active":"2024-03-19T01:52:09.490183+0000","last_peered":"2024-03-19T01:52:09.490183+0000","last_clean":"2024-03-19T01:52:09.490183+0000","last_became_active":"2024-03-19T01:50:50.557991+0000","last_became_peered":"2024-03-19T01:50:50.557991+0000","last_unstale":"2024-03-19T01:52:09.490183+0000","last_undegraded":"2024-03-19T01:52:09.490183+0000","last_fullsized":"2024-03-19T01:52:09.490183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:55:07.199605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.586157+0000","last_change":"2024-03-19T01:50:49.552491+0000","last_active":"2024-03-19T01:51:40.586157+0000","last_peered":"2024-03-19T01:51:40.586157+0000","last_clean":"2024-03-19T01:51:40.586157+0000","last_became_active":"2024-03-19T01:50:49.552133+0000","last_became_peered":"2024-03-19T01:50:49.552133+0000","last_unstale":"2024-03-19T01:51:40.586157+0000","last_undegraded":"2024-03-19T01:51:40.586157+0000","last_fullsized":"2024-03-19T01:51:40.586157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:47:44.458968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.493940+0000","last_change":"2024-03-19T01:51:40.590244+0000","last_active":"2024-03-19T01:52:09.493940+0000","last_peered":"2024-03-19T01:52:09.493940+0000","last_clean":"2024-03-19T01:52:09.493940+0000","last_became_active":"2024-03-19T01:51:40.589898+0000","last_became_peered":"2024-03-19T01:51:40.589898+0000","last_unstale":"2024-03-19T01:52:09.493940+0000","last_undegraded":"2024-03-19T01:52:09.493940+0000","last_fullsized":"2024-03-19T01:52:09.493940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:16:30.244828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587559+0000","last_change":"2024-03-19T01:50:49.549586+0000","last_active":"2024-03-19T01:51:40.587559+0000","last_peered":"2024-03-19T01:51:40.587559+0000","last_clean":"2024-03-19T01:51:40.587559+0000","last_became_active":"2024-03-19T01:50:49.549224+0000","last_became_peered":"2024-03-19T01:50:49.549224+0000","last_unstale":"2024-03-19T01:51:40.587559+0000","last_undegraded":"2024-03-19T01:51:40.587559+0000","last_fullsized":"2024-03-19T01:51:40.587559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:24:30.591771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.476955+0000","last_change":"2024-03-19T01:50:50.554120+0000","last_active":"2024-03-19T01:52:09.476955+0000","last_peered":"2024-03-19T01:52:09.476955+0000","last_clean":"2024-03-19T01:52:09.476955+0000","last_became_active":"2024-03-19T01:50:50.553636+0000","last_became_peered":"2024-03-19T01:50:50.553636+0000","last_unstale":"2024-03-19T01:52:09.476955+0000","last_undegraded":"2024-03-19T01:52:09.476955+0000","last_fullsized":"2024-03-19T01:52:09.476955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:47:27.634688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":99,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.055838+0000","last_change":"2024-03-19T01:50:49.551532+0000","last_active":"2024-03-19T01:52:09.055838+0000","last_peered":"2024-03-19T01:52:09.055838+0000","last_clean":"2024-03-19T01:52:09.055838+0000","last_became_active":"2024-03-19T01:50:49.551060+0000","last_became_peered":"2024-03-19T01:50:49.551060+0000","last_unstale":"2024-03-19T01:52:09.055838+0000","last_undegraded":"2024-03-19T01:52:09.055838+0000","last_fullsized":"2024-03-19T01:52:09.055838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:47:00.339785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.483255+0000","last_change":"2024-03-19T01:50:50.556606+0000","last_active":"2024-03-19T01:52:09.483255+0000","last_peered":"2024-03-19T01:52:09.483255+0000","last_clean":"2024-03-19T01:52:09.483255+0000","last_became_active":"2024-03-19T01:50:50.556089+0000","last_became_peered":"2024-03-19T01:50:50.556089+0000","last_unstale":"2024-03-19T01:52:09.483255+0000","last_undegraded":"2024-03-19T01:52:09.483255+0000","last_fullsized":"2024-03-19T01:52:09.483255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:42:24.256326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.493106+0000","last_change":"2024-03-19T01:50:50.564926+0000","last_active":"2024-03-19T01:52:09.493106+0000","last_peered":"2024-03-19T01:52:09.493106+0000","last_clean":"2024-03-19T01:52:09.493106+0000","last_became_active":"2024-03-19T01:50:50.564612+0000","last_became_peered":"2024-03-19T01:50:50.564612+0000","last_unstale":"2024-03-19T01:52:09.493106+0000","last_undegraded":"2024-03-19T01:52:09.493106+0000","last_fullsized":"2024-03-19T01:52:09.493106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:42:29.159512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:07.044189+0000","last_change":"2024-03-19T01:50:49.553780+0000","last_active":"2024-03-19T01:52:07.044189+0000","last_peered":"2024-03-19T01:52:07.044189+0000","last_clean":"2024-03-19T01:52:07.044189+0000","last_became_active":"2024-03-19T01:50:49.552795+0000","last_became_peered":"2024-03-19T01:50:49.552795+0000","last_unstale":"2024-03-19T01:52:07.044189+0000","last_undegraded":"2024-03-19T01:52:07.044189+0000","last_fullsized":"2024-03-19T01:52:07.044189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:01:37.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.493813+0000","last_change":"2024-03-19T01:50:50.556943+0000","last_active":"2024-03-19T01:52:09.493813+0000","last_peered":"2024-03-19T01:52:09.493813+0000","last_clean":"2024-03-19T01:52:09.493813+0000","last_became_active":"2024-03-19T01:50:50.556646+0000","last_became_peered":"2024-03-19T01:50:50.556646+0000","last_unstale":"2024-03-19T01:52:09.493813+0000","last_undegraded":"2024-03-19T01:52:09.493813+0000","last_fullsized":"2024-03-19T01:52:09.493813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:59:28.712332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.852431+0000","last_change":"2024-03-19T01:50:49.549760+0000","last_active":"2024-03-19T01:51:40.852431+0000","last_peered":"2024-03-19T01:51:40.852431+0000","last_clean":"2024-03-19T01:51:40.852431+0000","last_became_active":"2024-03-19T01:50:49.549218+0000","last_became_peered":"2024-03-19T01:50:49.549218+0000","last_unstale":"2024-03-19T01:51:40.852431+0000","last_undegraded":"2024-03-19T01:51:40.852431+0000","last_fullsized":"2024-03-19T01:51:40.852431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:30:00.802902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584830+0000","last_change":"2024-03-19T01:49:36.469079+0000","last_active":"2024-03-19T01:51:40.584830+0000","last_peered":"2024-03-19T01:51:40.584830+0000","last_clean":"2024-03-19T01:51:40.584830+0000","last_became_active":"2024-03-19T01:49:36.468624+0000","last_became_peered":"2024-03-19T01:49:36.468624+0000","last_unstale":"2024-03-19T01:51:40.584830+0000","last_undegraded":"2024-03-19T01:51:40.584830+0000","last_fullsized":"2024-03-19T01:51:40.584830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:46:07.695833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:46:07.695833+0000","last_clean_scrub_stamp":"2024-03-19T01:46:07.695833+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:06:31.554853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.704775+0000","last_change":"2024-03-19T01:50:50.550851+0000","last_active":"2024-03-19T01:51:40.704775+0000","last_peered":"2024-03-19T01:51:40.704775+0000","last_clean":"2024-03-19T01:51:40.704775+0000","last_became_active":"2024-03-19T01:50:50.550581+0000","last_became_peered":"2024-03-19T01:50:50.550581+0000","last_unstale":"2024-03-19T01:51:40.704775+0000","last_undegraded":"2024-03-19T01:51:40.704775+0000","last_fullsized":"2024-03-19T01:51:40.704775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:24:24.575915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584919+0000","last_change":"2024-03-19T01:50:49.554043+0000","last_active":"2024-03-19T01:51:40.584919+0000","last_peered":"2024-03-19T01:51:40.584919+0000","last_clean":"2024-03-19T01:51:40.584919+0000","last_became_active":"2024-03-19T01:50:49.553597+0000","last_became_peered":"2024-03-19T01:50:49.553597+0000","last_unstale":"2024-03-19T01:51:40.584919+0000","last_undegraded":"2024-03-19T01:51:40.584919+0000","last_fullsized":"2024-03-19T01:51:40.584919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:41:40.545382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.586219+0000","last_change":"2024-03-19T01:50:49.553740+0000","last_active":"2024-03-19T01:51:40.586219+0000","last_peered":"2024-03-19T01:51:40.586219+0000","last_clean":"2024-03-19T01:51:40.586219+0000","last_became_active":"2024-03-19T01:50:49.553230+0000","last_became_peered":"2024-03-19T01:50:49.553230+0000","last_unstale":"2024-03-19T01:51:40.586219+0000","last_undegraded":"2024-03-19T01:51:40.586219+0000","last_fullsized":"2024-03-19T01:51:40.586219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:46:36.633880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705069+0000","last_change":"2024-03-19T01:50:50.554478+0000","last_active":"2024-03-19T01:51:40.705069+0000","last_peered":"2024-03-19T01:51:40.705069+0000","last_clean":"2024-03-19T01:51:40.705069+0000","last_became_active":"2024-03-19T01:50:50.551170+0000","last_became_peered":"2024-03-19T01:50:50.551170+0000","last_unstale":"2024-03-19T01:51:40.705069+0000","last_undegraded":"2024-03-19T01:51:40.705069+0000","last_fullsized":"2024-03-19T01:51:40.705069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T12:08:12.382239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'13","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.588400+0000","last_change":"2024-03-19T01:50:49.554018+0000","last_active":"2024-03-19T01:51:40.588400+0000","last_peered":"2024-03-19T01:51:40.588400+0000","last_clean":"2024-03-19T01:51:40.588400+0000","last_became_active":"2024-03-19T01:50:49.553702+0000","last_became_peered":"2024-03-19T01:50:49.553702+0000","last_unstale":"2024-03-19T01:51:40.588400+0000","last_undegraded":"2024-03-19T01:51:40.588400+0000","last_fullsized":"2024-03-19T01:51:40.588400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:15:55.051901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.490692+0000","last_change":"2024-03-19T01:50:50.565955+0000","last_active":"2024-03-19T01:52:09.490692+0000","last_peered":"2024-03-19T01:52:09.490692+0000","last_clean":"2024-03-19T01:52:09.490692+0000","last_became_active":"2024-03-19T01:50:50.565643+0000","last_became_peered":"2024-03-19T01:50:50.565643+0000","last_unstale":"2024-03-19T01:52:09.490692+0000","last_undegraded":"2024-03-19T01:52:09.490692+0000","last_fullsized":"2024-03-19T01:52:09.490692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:37:42.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.852765+0000","last_change":"2024-03-19T01:50:49.548298+0000","last_active":"2024-03-19T01:51:40.852765+0000","last_peered":"2024-03-19T01:51:40.852765+0000","last_clean":"2024-03-19T01:51:40.852765+0000","last_became_active":"2024-03-19T01:50:49.547941+0000","last_became_peered":"2024-03-19T01:50:49.547941+0000","last_unstale":"2024-03-19T01:51:40.852765+0000","last_undegraded":"2024-03-19T01:51:40.852765+0000","last_fullsized":"2024-03-19T01:51:40.852765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:23:59.221152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584731+0000","last_change":"2024-03-19T01:50:50.563923+0000","last_active":"2024-03-19T01:51:40.584731+0000","last_peered":"2024-03-19T01:51:40.584731+0000","last_clean":"2024-03-19T01:51:40.584731+0000","last_became_active":"2024-03-19T01:50:50.563295+0000","last_became_peered":"2024-03-19T01:50:50.563295+0000","last_unstale":"2024-03-19T01:51:40.584731+0000","last_undegraded":"2024-03-19T01:51:40.584731+0000","last_fullsized":"2024-03-19T01:51:40.584731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T13:09:47.089720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590012+0000","last_change":"2024-03-19T01:50:49.548014+0000","last_active":"2024-03-19T01:51:40.590012+0000","last_peered":"2024-03-19T01:51:40.590012+0000","last_clean":"2024-03-19T01:51:40.590012+0000","last_became_active":"2024-03-19T01:50:49.547588+0000","last_became_peered":"2024-03-19T01:50:49.547588+0000","last_unstale":"2024-03-19T01:51:40.590012+0000","last_undegraded":"2024-03-19T01:51:40.590012+0000","last_fullsized":"2024-03-19T01:51:40.590012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:23:08.258554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.484798+0000","last_change":"2024-03-19T01:50:50.562437+0000","last_active":"2024-03-19T01:52:09.484798+0000","last_peered":"2024-03-19T01:52:09.484798+0000","last_clean":"2024-03-19T01:52:09.484798+0000","last_became_active":"2024-03-19T01:50:50.562113+0000","last_became_peered":"2024-03-19T01:50:50.562113+0000","last_unstale":"2024-03-19T01:52:09.484798+0000","last_undegraded":"2024-03-19T01:52:09.484798+0000","last_fullsized":"2024-03-19T01:52:09.484798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:19:31.932037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587239+0000","last_change":"2024-03-19T01:50:49.547190+0000","last_active":"2024-03-19T01:51:40.587239+0000","last_peered":"2024-03-19T01:51:40.587239+0000","last_clean":"2024-03-19T01:51:40.587239+0000","last_became_active":"2024-03-19T01:50:49.546698+0000","last_became_peered":"2024-03-19T01:50:49.546698+0000","last_unstale":"2024-03-19T01:51:40.587239+0000","last_undegraded":"2024-03-19T01:51:40.587239+0000","last_fullsized":"2024-03-19T01:51:40.587239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:09.557035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.494170+0000","last_change":"2024-03-19T01:50:50.571159+0000","last_active":"2024-03-19T01:52:09.494170+0000","last_peered":"2024-03-19T01:52:09.494170+0000","last_clean":"2024-03-19T01:52:09.494170+0000","last_became_active":"2024-03-19T01:50:50.570312+0000","last_became_peered":"2024-03-19T01:50:50.570312+0000","last_unstale":"2024-03-19T01:52:09.494170+0000","last_undegraded":"2024-03-19T01:52:09.494170+0000","last_fullsized":"2024-03-19T01:52:09.494170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:23:16.820114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590785+0000","last_change":"2024-03-19T01:50:49.550557+0000","last_active":"2024-03-19T01:51:40.590785+0000","last_peered":"2024-03-19T01:51:40.590785+0000","last_clean":"2024-03-19T01:51:40.590785+0000","last_became_active":"2024-03-19T01:50:49.549789+0000","last_became_peered":"2024-03-19T01:50:49.549789+0000","last_unstale":"2024-03-19T01:51:40.590785+0000","last_undegraded":"2024-03-19T01:51:40.590785+0000","last_fullsized":"2024-03-19T01:51:40.590785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:06:14.292689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.478960+0000","last_change":"2024-03-19T01:50:50.560116+0000","last_active":"2024-03-19T01:52:09.478960+0000","last_peered":"2024-03-19T01:52:09.478960+0000","last_clean":"2024-03-19T01:52:09.478960+0000","last_became_active":"2024-03-19T01:50:50.559809+0000","last_became_peered":"2024-03-19T01:50:50.559809+0000","last_unstale":"2024-03-19T01:52:09.478960+0000","last_undegraded":"2024-03-19T01:52:09.478960+0000","last_fullsized":"2024-03-19T01:52:09.478960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:27:26.854262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.585620+0000","last_change":"2024-03-19T01:50:49.556100+0000","last_active":"2024-03-19T01:51:40.585620+0000","last_peered":"2024-03-19T01:51:40.585620+0000","last_clean":"2024-03-19T01:51:40.585620+0000","last_became_active":"2024-03-19T01:50:49.555342+0000","last_became_peered":"2024-03-19T01:50:49.555342+0000","last_unstale":"2024-03-19T01:51:40.585620+0000","last_undegraded":"2024-03-19T01:51:40.585620+0000","last_fullsized":"2024-03-19T01:51:40.585620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:50:15.712384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.492564+0000","last_change":"2024-03-19T01:50:50.552792+0000","last_active":"2024-03-19T01:52:09.492564+0000","last_peered":"2024-03-19T01:52:09.492564+0000","last_clean":"2024-03-19T01:52:09.492564+0000","last_became_active":"2024-03-19T01:50:50.552438+0000","last_became_peered":"2024-03-19T01:50:50.552438+0000","last_unstale":"2024-03-19T01:52:09.492564+0000","last_undegraded":"2024-03-19T01:52:09.492564+0000","last_fullsized":"2024-03-19T01:52:09.492564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:42:10.436248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:11.048787+0000","last_change":"2024-03-19T01:50:49.546948+0000","last_active":"2024-03-19T01:52:11.048787+0000","last_peered":"2024-03-19T01:52:11.048787+0000","last_clean":"2024-03-19T01:52:11.048787+0000","last_became_active":"2024-03-19T01:50:49.546610+0000","last_became_peered":"2024-03-19T01:50:49.546610+0000","last_unstale":"2024-03-19T01:52:11.048787+0000","last_undegraded":"2024-03-19T01:52:11.048787+0000","last_fullsized":"2024-03-19T01:52:11.048787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:06:09.973827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.492413+0000","last_change":"2024-03-19T01:50:50.552703+0000","last_active":"2024-03-19T01:52:09.492413+0000","last_peered":"2024-03-19T01:52:09.492413+0000","last_clean":"2024-03-19T01:52:09.492413+0000","last_became_active":"2024-03-19T01:50:50.552337+0000","last_became_peered":"2024-03-19T01:50:50.552337+0000","last_unstale":"2024-03-19T01:52:09.492413+0000","last_undegraded":"2024-03-19T01:52:09.492413+0000","last_fullsized":"2024-03-19T01:52:09.492413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T10:14:27.912513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.587171+0000","last_change":"2024-03-19T01:50:49.554531+0000","last_active":"2024-03-19T01:51:40.587171+0000","last_peered":"2024-03-19T01:51:40.587171+0000","last_clean":"2024-03-19T01:51:40.587171+0000","last_became_active":"2024-03-19T01:50:49.554196+0000","last_became_peered":"2024-03-19T01:50:49.554196+0000","last_unstale":"2024-03-19T01:51:40.587171+0000","last_undegraded":"2024-03-19T01:51:40.587171+0000","last_fullsized":"2024-03-19T01:51:40.587171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:01:04.650010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.480677+0000","last_change":"2024-03-19T01:51:40.593555+0000","last_active":"2024-03-19T01:52:09.480677+0000","last_peered":"2024-03-19T01:52:09.480677+0000","last_clean":"2024-03-19T01:52:09.480677+0000","last_became_active":"2024-03-19T01:51:40.589104+0000","last_became_peered":"2024-03-19T01:51:40.589104+0000","last_unstale":"2024-03-19T01:52:09.480677+0000","last_undegraded":"2024-03-19T01:52:09.480677+0000","last_fullsized":"2024-03-19T01:52:09.480677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:25:44.319156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705010+0000","last_change":"2024-03-19T01:50:49.553203+0000","last_active":"2024-03-19T01:51:40.705010+0000","last_peered":"2024-03-19T01:51:40.705010+0000","last_clean":"2024-03-19T01:51:40.705010+0000","last_became_active":"2024-03-19T01:50:49.552243+0000","last_became_peered":"2024-03-19T01:50:49.552243+0000","last_unstale":"2024-03-19T01:51:40.705010+0000","last_undegraded":"2024-03-19T01:51:40.705010+0000","last_fullsized":"2024-03-19T01:51:40.705010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:48.515318+0000","last_clean_scrub_stamp":"2024-03-19T01:50:48.515318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:41:16.313075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.490906+0000","last_change":"2024-03-19T01:50:50.567317+0000","last_active":"2024-03-19T01:52:09.490906+0000","last_peered":"2024-03-19T01:52:09.490906+0000","last_clean":"2024-03-19T01:52:09.490906+0000","last_became_active":"2024-03-19T01:50:50.566744+0000","last_became_peered":"2024-03-19T01:50:50.566744+0000","last_unstale":"2024-03-19T01:52:09.490906+0000","last_undegraded":"2024-03-19T01:52:09.490906+0000","last_fullsized":"2024-03-19T01:52:09.490906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:01:34.239148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.494273+0000","last_change":"2024-03-19T01:51:40.592726+0000","last_active":"2024-03-19T01:52:09.494273+0000","last_peered":"2024-03-19T01:52:09.494273+0000","last_clean":"2024-03-19T01:52:09.494273+0000","last_became_active":"2024-03-19T01:51:40.592375+0000","last_became_peered":"2024-03-19T01:51:40.592375+0000","last_unstale":"2024-03-19T01:52:09.494273+0000","last_undegraded":"2024-03-19T01:52:09.494273+0000","last_fullsized":"2024-03-19T01:52:09.494273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:05:06.193052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.488851+0000","last_change":"2024-03-19T01:50:50.564337+0000","last_active":"2024-03-19T01:52:09.488851+0000","last_peered":"2024-03-19T01:52:09.488851+0000","last_clean":"2024-03-19T01:52:09.488851+0000","last_became_active":"2024-03-19T01:50:50.564002+0000","last_became_peered":"2024-03-19T01:50:50.564002+0000","last_unstale":"2024-03-19T01:52:09.488851+0000","last_undegraded":"2024-03-19T01:52:09.488851+0000","last_fullsized":"2024-03-19T01:52:09.488851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T11:19:51.072829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.483143+0000","last_change":"2024-03-19T01:50:50.560962+0000","last_active":"2024-03-19T01:52:09.483143+0000","last_peered":"2024-03-19T01:52:09.483143+0000","last_clean":"2024-03-19T01:52:09.483143+0000","last_became_active":"2024-03-19T01:50:50.560427+0000","last_became_peered":"2024-03-19T01:50:50.560427+0000","last_unstale":"2024-03-19T01:52:09.483143+0000","last_undegraded":"2024-03-19T01:52:09.483143+0000","last_fullsized":"2024-03-19T01:52:09.483143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:43:54.472200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.584331+0000","last_change":"2024-03-19T01:50:50.557806+0000","last_active":"2024-03-19T01:51:40.584331+0000","last_peered":"2024-03-19T01:51:40.584331+0000","last_clean":"2024-03-19T01:51:40.584331+0000","last_became_active":"2024-03-19T01:50:50.557494+0000","last_became_peered":"2024-03-19T01:50:50.557494+0000","last_unstale":"2024-03-19T01:51:40.584331+0000","last_undegraded":"2024-03-19T01:51:40.584331+0000","last_fullsized":"2024-03-19T01:51:40.584331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:35:53.787047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705161+0000","last_change":"2024-03-19T01:50:50.554414+0000","last_active":"2024-03-19T01:51:40.705161+0000","last_peered":"2024-03-19T01:51:40.705161+0000","last_clean":"2024-03-19T01:51:40.705161+0000","last_became_active":"2024-03-19T01:50:50.554010+0000","last_became_peered":"2024-03-19T01:50:50.554010+0000","last_unstale":"2024-03-19T01:51:40.705161+0000","last_undegraded":"2024-03-19T01:51:40.705161+0000","last_fullsized":"2024-03-19T01:51:40.705161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:02:54.193715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.494700+0000","last_change":"2024-03-19T01:50:50.567759+0000","last_active":"2024-03-19T01:52:09.494700+0000","last_peered":"2024-03-19T01:52:09.494700+0000","last_clean":"2024-03-19T01:52:09.494700+0000","last_became_active":"2024-03-19T01:50:50.566529+0000","last_became_peered":"2024-03-19T01:50:50.566529+0000","last_unstale":"2024-03-19T01:52:09.494700+0000","last_undegraded":"2024-03-19T01:52:09.494700+0000","last_fullsized":"2024-03-19T01:52:09.494700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:25:05.002161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.496446+0000","last_change":"2024-03-19T01:50:50.563915+0000","last_active":"2024-03-19T01:52:09.496446+0000","last_peered":"2024-03-19T01:52:09.496446+0000","last_clean":"2024-03-19T01:52:09.496446+0000","last_became_active":"2024-03-19T01:50:50.563596+0000","last_became_peered":"2024-03-19T01:50:50.563596+0000","last_unstale":"2024-03-19T01:52:09.496446+0000","last_undegraded":"2024-03-19T01:52:09.496446+0000","last_fullsized":"2024-03-19T01:52:09.496446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:35:12.832402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.483298+0000","last_change":"2024-03-19T01:50:50.556507+0000","last_active":"2024-03-19T01:52:09.483298+0000","last_peered":"2024-03-19T01:52:09.483298+0000","last_clean":"2024-03-19T01:52:09.483298+0000","last_became_active":"2024-03-19T01:50:50.556197+0000","last_became_peered":"2024-03-19T01:50:50.556197+0000","last_unstale":"2024-03-19T01:52:09.483298+0000","last_undegraded":"2024-03-19T01:52:09.483298+0000","last_fullsized":"2024-03-19T01:52:09.483298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:25:26.919274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.488866+0000","last_change":"2024-03-19T01:50:50.554403+0000","last_active":"2024-03-19T01:52:09.488866+0000","last_peered":"2024-03-19T01:52:09.488866+0000","last_clean":"2024-03-19T01:52:09.488866+0000","last_became_active":"2024-03-19T01:50:50.553985+0000","last_became_peered":"2024-03-19T01:50:50.553985+0000","last_unstale":"2024-03-19T01:52:09.488866+0000","last_undegraded":"2024-03-19T01:52:09.488866+0000","last_fullsized":"2024-03-19T01:52:09.488866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:25:50.923915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.483713+0000","last_change":"2024-03-19T01:50:50.554391+0000","last_active":"2024-03-19T01:52:09.483713+0000","last_peered":"2024-03-19T01:52:09.483713+0000","last_clean":"2024-03-19T01:52:09.483713+0000","last_became_active":"2024-03-19T01:50:50.552987+0000","last_became_peered":"2024-03-19T01:50:50.552987+0000","last_unstale":"2024-03-19T01:52:09.483713+0000","last_undegraded":"2024-03-19T01:52:09.483713+0000","last_fullsized":"2024-03-19T01:52:09.483713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:17:07.107613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.590286+0000","last_change":"2024-03-19T01:50:50.556172+0000","last_active":"2024-03-19T01:51:40.590286+0000","last_peered":"2024-03-19T01:51:40.590286+0000","last_clean":"2024-03-19T01:51:40.590286+0000","last_became_active":"2024-03-19T01:50:50.555468+0000","last_became_peered":"2024-03-19T01:50:50.555468+0000","last_unstale":"2024-03-19T01:51:40.590286+0000","last_undegraded":"2024-03-19T01:51:40.590286+0000","last_fullsized":"2024-03-19T01:51:40.590286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:28:03.005962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:51:40.705205+0000","last_change":"2024-03-19T01:50:50.550915+0000","last_active":"2024-03-19T01:51:40.705205+0000","last_peered":"2024-03-19T01:51:40.705205+0000","last_clean":"2024-03-19T01:51:40.705205+0000","last_became_active":"2024-03-19T01:50:50.550535+0000","last_became_peered":"2024-03-19T01:50:50.550535+0000","last_unstale":"2024-03-19T01:51:40.705205+0000","last_undegraded":"2024-03-19T01:51:40.705205+0000","last_fullsized":"2024-03-19T01:51:40.705205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:05:21.959483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-19T01:52:09.494919+0000","last_change":"2024-03-19T01:50:50.567475+0000","last_active":"2024-03-19T01:52:09.494919+0000","last_peered":"2024-03-19T01:52:09.494919+0000","last_clean":"2024-03-19T01:52:09.494919+0000","last_became_active":"2024-03-19T01:50:50.567103+0000","last_became_peered":"2024-03-19T01:50:50.567103+0000","last_unstale":"2024-03-19T01:52:09.494919+0000","last_undegraded":"2024-03-19T01:52:09.494919+0000","last_fullsized":"2024-03-19T01:52:09.494919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-19T01:50:49.524625+0000","last_clean_scrub_stamp":"2024-03-19T01:50:49.524625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T09:29:41.866610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244932,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66668,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":805,"num_read_kb":475,"num_write":243,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":650,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":199755,"data_compressed":5493,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383710,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":457449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645348,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1107542,"data_compressed":37525,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":456846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":456525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1049766,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":516824,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":457785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":517757,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":456154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1049444,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34052,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35694,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61217,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62150,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61450,"data_compressed":1831,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-19T01:52:14.474 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-19T01:52:14.474 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-19T01:52:14.475 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-19T01:52:14.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-19T01:52:14.720 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:14.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:14 smithi092 ceph-mon[62574]: from='client.24817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:52:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:14 smithi159 ceph-mon[67110]: from='client.24817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:52:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:14 smithi144 ceph-mon[67199]: from='client.24817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-19T01:52:15.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:15 smithi159 ceph-mon[67110]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:52:15.956 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:15 smithi092 ceph-mon[62574]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:52:16.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:15 smithi144 ceph-mon[67199]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:52:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.511+0000 7f50872af700 1 -- 172.21.15.92:0/3032957319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5080101e50 msgr2=0x7f5080105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.511+0000 7f50872af700 1 --2- 172.21.15.92:0/3032957319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5080101e50 0x7f5080105e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f50700099c0 tx=0x7f5070009cd0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.511+0000 7f50872af700 1 -- 172.21.15.92:0/3032957319 shutdown_connections 2024-03-19T01:52:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.511+0000 7f50872af700 1 --2- 172.21.15.92:0/3032957319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50801064f0 0x7f508010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.511+0000 7f50872af700 1 --2- 172.21.15.92:0/3032957319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50801014a0 0x7f5080101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.511+0000 7f50872af700 1 --2- 172.21.15.92:0/3032957319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5080101e50 0x7f5080105e10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.511+0000 7f50872af700 1 -- 172.21.15.92:0/3032957319 >> 172.21.15.92:0/3032957319 conn(0x7f5080077260 msgr2=0x7f5080077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.511+0000 7f50872af700 1 -- 172.21.15.92:0/3032957319 shutdown_connections 2024-03-19T01:52:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.511+0000 7f50872af700 1 -- 172.21.15.92:0/3032957319 wait complete. 2024-03-19T01:52:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.512+0000 7f50872af700 1 Processor -- start 2024-03-19T01:52:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.512+0000 7f50872af700 1 -- start start 2024-03-19T01:52:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.512+0000 7f50872af700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50801014a0 0x7f5080199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.512+0000 7f50872af700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5080101e50 0x7f5080199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.512+0000 7f50872af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50801064f0 0x7f50801a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.512+0000 7f50872af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5080111500 con 0x7f50801064f0 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.512+0000 7f50872af700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5080111380 con 0x7f50801014a0 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.512+0000 7f50872af700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5080111680 con 0x7f5080101e50 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508584c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50801064f0 0x7f50801a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508484a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5080101e50 0x7f5080199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508504b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50801014a0 0x7f5080199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508584c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50801064f0 0x7f50801a4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36754/0 (socket says 172.21.15.92:36754) 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508584c700 1 -- 172.21.15.92:0/4112690245 learned_addr learned my addr 172.21.15.92:0/4112690245 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508484a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5080101e50 0x7f5080199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:47398/0 (socket says 172.21.15.92:47398) 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508484a700 1 -- 172.21.15.92:0/4112690245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50801014a0 msgr2=0x7f5080199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508484a700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50801014a0 0x7f5080199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.515 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508484a700 1 -- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50801064f0 msgr2=0x7f50801a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508484a700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50801064f0 0x7f50801a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508484a700 1 -- 172.21.15.92:0/4112690245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50801a4f40 con 0x7f5080101e50 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508584c700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50801064f0 0x7f50801a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508504b700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50801014a0 0x7f5080199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f508484a700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5080101e50 0x7f5080199ec0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f50700098e0 tx=0x7f507000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f50767fc700 1 -- 172.21.15.92:0/4112690245 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070027070 con 0x7f5080101e50 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5080197770 con 0x7f5080101e50 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f50767fc700 1 -- 172.21.15.92:0/4112690245 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5070003680 con 0x7f5080101e50 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f50767fc700 1 -- 172.21.15.92:0/4112690245 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50700173f0 con 0x7f5080101e50 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.513+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5080197ca0 con 0x7f5080101e50 2024-03-19T01:52:16.516 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.514+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5080068fd0 con 0x7f5080101e50 2024-03-19T01:52:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.516+0000 7f50767fc700 1 -- 172.21.15.92:0/4112690245 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f507002a070 con 0x7f5080101e50 2024-03-19T01:52:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.516+0000 7f50767fc700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f506c0a2910 0x7f506c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.516+0000 7f508504b700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f506c0a2910 0x7f506c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.516+0000 7f50767fc700 1 -- 172.21.15.92:0/4112690245 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f50700cf6c0 con 0x7f5080101e50 2024-03-19T01:52:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.517+0000 7f508504b700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f506c0a2910 0x7f506c0a4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f507c011490 tx=0x7f507c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:16.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.520+0000 7f50767fc700 1 -- 172.21.15.92:0/4112690245 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5070013070 con 0x7f5080101e50 2024-03-19T01:52:16.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.768+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f508006aea0 con 0x7f5080101e50 2024-03-19T01:52:16.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.768+0000 7f50767fc700 1 -- 172.21.15.92:0/4112690245 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f507009c7a0 con 0x7f5080101e50 2024-03-19T01:52:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:16.769 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-19T01:52:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f506c0a2910 msgr2=0x7f506c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f506c0a2910 0x7f506c0a4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f507c011490 tx=0x7f507c009260 comp rx=0 tx=0).stop 2024-03-19T01:52:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5080101e50 msgr2=0x7f5080199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5080101e50 0x7f5080199ec0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f50700098e0 tx=0x7f507000fa00 comp rx=0 tx=0).stop 2024-03-19T01:52:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 shutdown_connections 2024-03-19T01:52:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f506c0a2910 0x7f506c0a4dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50801064f0 0x7f50801a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50801014a0 0x7f5080199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 --2- 172.21.15.92:0/4112690245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5080101e50 0x7f5080199ec0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 >> 172.21.15.92:0/4112690245 conn(0x7f5080077260 msgr2=0x7f50800ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 shutdown_connections 2024-03-19T01:52:16.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:16.771+0000 7f50872af700 1 -- 172.21.15.92:0/4112690245 wait complete. 2024-03-19T01:52:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:16 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4112690245' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-19T01:52:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:16 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4112690245' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-19T01:52:17.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:16 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4112690245' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-19T01:52:17.517 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-19T01:52:17.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:52:17.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.622+0000 7f9d887c4640 1 -- 172.21.15.92:0/3862629331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 msgr2=0x7f9d80108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:17.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.622+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3862629331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 0x7f9d80108b80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009ca0 tx=0x7f9d7802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.622+0000 7f9d887c4640 1 -- 172.21.15.92:0/3862629331 shutdown_connections 2024-03-19T01:52:17.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.622+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3862629331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 0x7f9d80108b80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.622+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3862629331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d80101930 0x7f9d80101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.622+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3862629331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d80100740 0x7f9d80100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.622+0000 7f9d887c4640 1 -- 172.21.15.92:0/3862629331 >> 172.21.15.92:0/3862629331 conn(0x7f9d800fbcb0 msgr2=0x7f9d800fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:17.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.622+0000 7f9d887c4640 1 -- 172.21.15.92:0/3862629331 shutdown_connections 2024-03-19T01:52:17.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.622+0000 7f9d887c4640 1 -- 172.21.15.92:0/3862629331 wait complete. 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d887c4640 1 Processor -- start 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d887c4640 1 -- start start 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d887c4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d80100740 0x7f9d80197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d887c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d80101930 0x7f9d80198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d887c4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 0x7f9d8019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d887c4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8010c3c0 con 0x7f9d801022f0 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d887c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8010c240 con 0x7f9d80101930 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d887c4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8010c540 con 0x7f9d80100740 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d86d61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 0x7f9d8019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d86d61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 0x7f9d8019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36776/0 (socket says 172.21.15.92:36776) 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d86d61640 1 -- 172.21.15.92:0/3991274673 learned_addr learned my addr 172.21.15.92:0/3991274673 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d86560640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d80100740 0x7f9d80197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:17.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d85d5f640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d80101930 0x7f9d80198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d86d61640 1 -- 172.21.15.92:0/3991274673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d80100740 msgr2=0x7f9d80197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d86d61640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d80100740 0x7f9d80197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d86d61640 1 -- 172.21.15.92:0/3991274673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d80101930 msgr2=0x7f9d80198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d86d61640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d80101930 0x7f9d80198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d86d61640 1 -- 172.21.15.92:0/3991274673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8019f720 con 0x7f9d801022f0 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.623+0000 7f9d85d5f640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d80101930 0x7f9d80198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.624+0000 7f9d86d61640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 0x7f9d8019f0b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d78037040 tx=0x7f9d78032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.624+0000 7f9d777fe640 1 -- 172.21.15.92:0/3991274673 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78041020 con 0x7f9d801022f0 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.624+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8019f9b0 con 0x7f9d801022f0 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.624+0000 7f9d777fe640 1 -- 172.21.15.92:0/3991274673 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d78034cd0 con 0x7f9d801022f0 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.624+0000 7f9d777fe640 1 -- 172.21.15.92:0/3991274673 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78039040 con 0x7f9d801022f0 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.624+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8019fea0 con 0x7f9d801022f0 2024-03-19T01:52:17.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.625+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d801a00d0 con 0x7f9d801022f0 2024-03-19T01:52:17.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.626+0000 7f9d777fe640 1 -- 172.21.15.92:0/3991274673 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9d78044020 con 0x7f9d801022f0 2024-03-19T01:52:17.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.627+0000 7f9d777fe640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d6c0a2800 0x7f9d6c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:17.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.627+0000 7f9d86560640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d6c0a2800 0x7f9d6c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:17.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.627+0000 7f9d777fe640 1 -- 172.21.15.92:0/3991274673 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9d780f0de0 con 0x7f9d801022f0 2024-03-19T01:52:17.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.627+0000 7f9d86560640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d6c0a2800 0x7f9d6c0a4cc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9d80101790 tx=0x7f9d7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:17.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.630+0000 7f9d777fe640 1 -- 172.21.15.92:0/3991274673 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9d7804b020 con 0x7f9d801022f0 2024-03-19T01:52:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.851+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d801a0460 con 0x7f9d801022f0 2024-03-19T01:52:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.853+0000 7f9d777fe640 1 -- 172.21.15.92:0/3991274673 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22690 (secure 0 0 0) 0x7f9d780bdf20 con 0x7f9d801022f0 2024-03-19T01:52:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:17.854 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":87,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:51:40.581546+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:52:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d6c0a2800 msgr2=0x7f9d6c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d6c0a2800 0x7f9d6c0a4cc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9d80101790 tx=0x7f9d7c005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 msgr2=0x7f9d8019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 0x7f9d8019f0b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d78037040 tx=0x7f9d78032f40 comp rx=0 tx=0).stop 2024-03-19T01:52:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 shutdown_connections 2024-03-19T01:52:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d801022f0 0x7f9d8019f0b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d80101930 0x7f9d80198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d6c0a2800 0x7f9d6c0a4cc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 --2- 172.21.15.92:0/3991274673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d80100740 0x7f9d80197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 >> 172.21.15.92:0/3991274673 conn(0x7f9d800fbcb0 msgr2=0x7f9d800fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 shutdown_connections 2024-03-19T01:52:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.855+0000 7f9d887c4640 1 -- 172.21.15.92:0/3991274673 wait complete. 2024-03-19T01:52:17.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:46:07.695753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:50:47.888441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:50:49.068308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:52:17.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:52:17.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 -- 172.21.15.92:0/3527906574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 msgr2=0x7f1488101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:17.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 --2- 172.21.15.92:0/3527906574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 0x7f1488101d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f147c009900 tx=0x7f147c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:17.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 -- 172.21.15.92:0/3527906574 shutdown_connections 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 --2- 172.21.15.92:0/3527906574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14881022d0 0x7f1488108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 --2- 172.21.15.92:0/3527906574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 0x7f1488101d90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 --2- 172.21.15.92:0/3527906574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1488100720 0x7f1488100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 -- 172.21.15.92:0/3527906574 >> 172.21.15.92:0/3527906574 conn(0x7f14880fbcb0 msgr2=0x7f14880fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 -- 172.21.15.92:0/3527906574 shutdown_connections 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 -- 172.21.15.92:0/3527906574 wait complete. 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.972+0000 7f148deb2640 1 Processor -- start 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f148deb2640 1 -- start start 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f148deb2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1488100720 0x7f148810b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f148deb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 0x7f14881096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f148deb2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14881022d0 0x7f1488109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f148deb2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f148810d4c0 con 0x7f1488101910 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f148deb2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f148810d340 con 0x7f14881022d0 2024-03-19T01:52:17.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f148deb2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f148810d640 con 0x7f1488100720 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1486ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 0x7f14881096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1486ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 0x7f14881096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36802/0 (socket says 172.21.15.92:36802) 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1486ffd640 1 -- 172.21.15.92:0/2169034056 learned_addr learned my addr 172.21.15.92:0/2169034056 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f14877fe640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1488100720 0x7f148810b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1487fff640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14881022d0 0x7f1488109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1486ffd640 1 -- 172.21.15.92:0/2169034056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1488100720 msgr2=0x7f148810b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1486ffd640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1488100720 0x7f148810b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1486ffd640 1 -- 172.21.15.92:0/2169034056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14881022d0 msgr2=0x7f1488109c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1486ffd640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14881022d0 0x7f1488109c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1486ffd640 1 -- 172.21.15.92:0/2169034056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148810a4b0 con 0x7f1488101910 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.973+0000 7f1486ffd640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 0x7f14881096e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f147c009230 tx=0x7f147c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.974+0000 7f1484ff9640 1 -- 172.21.15.92:0/2169034056 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c041070 con 0x7f1488101910 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.974+0000 7f1484ff9640 1 -- 172.21.15.92:0/2169034056 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c004450 con 0x7f1488101910 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.974+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14881a1c00 con 0x7f1488101910 2024-03-19T01:52:17.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.974+0000 7f1484ff9640 1 -- 172.21.15.92:0/2169034056 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c003e20 con 0x7f1488101910 2024-03-19T01:52:17.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.974+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a2150 con 0x7f1488101910 2024-03-19T01:52:17.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.975+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1488107a60 con 0x7f1488101910 2024-03-19T01:52:17.977 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.976+0000 7f1484ff9640 1 -- 172.21.15.92:0/2169034056 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f147c044070 con 0x7f1488101910 2024-03-19T01:52:17.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.977+0000 7f1484ff9640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14700a2800 0x7f14700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:17.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.977+0000 7f14877fe640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14700a2800 0x7f14700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:17.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.977+0000 7f1484ff9640 1 -- 172.21.15.92:0/2169034056 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f147c0f05d0 con 0x7f1488101910 2024-03-19T01:52:17.978 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.977+0000 7f14877fe640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14700a2800 0x7f14700a4cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1488101770 tx=0x7f1478005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:17.981 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:17.981+0000 7f1484ff9640 1 -- 172.21.15.92:0/2169034056 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f147c0bd030 con 0x7f1488101910 2024-03-19T01:52:18.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:17 smithi144 ceph-mon[67199]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-19T01:52:18.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:17 smithi159 ceph-mon[67110]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-19T01:52:18.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.226+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1488062420 con 0x7f1488101910 2024-03-19T01:52:18.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.227+0000 7f1484ff9640 1 -- 172.21.15.92:0/2169034056 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f147c0c10d0 con 0x7f1488101910 2024-03-19T01:52:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-03-19T01:52:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.228+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14700a2800 msgr2=0x7f14700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.228+0000 7f148deb2640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14700a2800 0x7f14700a4cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1488101770 tx=0x7f1478005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 msgr2=0x7f14881096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 0x7f14881096e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f147c009230 tx=0x7f147c02c650 comp rx=0 tx=0).stop 2024-03-19T01:52:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 shutdown_connections 2024-03-19T01:52:18.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14881022d0 0x7f1488109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1488101910 0x7f14881096e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14700a2800 0x7f14700a4cc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 --2- 172.21.15.92:0/2169034056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1488100720 0x7f148810b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 >> 172.21.15.92:0/2169034056 conn(0x7f14880fbcb0 msgr2=0x7f1488104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 shutdown_connections 2024-03-19T01:52:18.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.229+0000 7f148deb2640 1 -- 172.21.15.92:0/2169034056 wait complete. 2024-03-19T01:52:18.239 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:17 smithi092 ceph-mon[62574]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-19T01:52:18.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:52:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.348+0000 7f3200e31640 1 -- 172.21.15.92:0/4167975453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31fc101850 msgr2=0x7f31fc101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.348+0000 7f3200e31640 1 --2- 172.21.15.92:0/4167975453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31fc101850 0x7f31fc101c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009860 tx=0x7f31ec02bb70 comp rx=0 tx=0).stop 2024-03-19T01:52:18.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.348+0000 7f3200e31640 1 -- 172.21.15.92:0/4167975453 shutdown_connections 2024-03-19T01:52:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.348+0000 7f3200e31640 1 --2- 172.21.15.92:0/4167975453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc103400 0x7f31fc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.348+0000 7f3200e31640 1 --2- 172.21.15.92:0/4167975453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31fc102a40 0x7f31fc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.348+0000 7f3200e31640 1 --2- 172.21.15.92:0/4167975453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31fc101850 0x7f31fc101c70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.348+0000 7f3200e31640 1 -- 172.21.15.92:0/4167975453 >> 172.21.15.92:0/4167975453 conn(0x7f31fc0fcdc0 msgr2=0x7f31fc0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.348+0000 7f3200e31640 1 -- 172.21.15.92:0/4167975453 shutdown_connections 2024-03-19T01:52:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.348+0000 7f3200e31640 1 -- 172.21.15.92:0/4167975453 wait complete. 2024-03-19T01:52:18.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f3200e31640 1 Processor -- start 2024-03-19T01:52:18.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f3200e31640 1 -- start start 2024-03-19T01:52:18.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f3200e31640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31fc101850 0x7f31fc197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f3200e31640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31fc102a40 0x7f31fc1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f3200e31640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc103400 0x7f31fc19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f3200e31640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc10d4c0 con 0x7f31fc103400 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f3200e31640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc10d340 con 0x7f31fc102a40 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f3200e31640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc10d640 con 0x7f31fc101850 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc103400 0x7f31fc19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc103400 0x7f31fc19f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36814/0 (socket says 172.21.15.92:36814) 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fad9d640 1 -- 172.21.15.92:0/3861688005 learned_addr learned my addr 172.21.15.92:0/3861688005 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fa59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31fc101850 0x7f31fc197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31f3fff640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31fc102a40 0x7f31fc1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fa59c640 1 -- 172.21.15.92:0/3861688005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31fc102a40 msgr2=0x7f31fc1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fa59c640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31fc102a40 0x7f31fc1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fa59c640 1 -- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc103400 msgr2=0x7f31fc19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fa59c640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc103400 0x7f31fc19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fa59c640 1 -- 172.21.15.92:0/3861688005 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc19f7c0 con 0x7f31fc101850 2024-03-19T01:52:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.349+0000 7f31fad9d640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc103400 0x7f31fc19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.350+0000 7f31fa59c640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31fc101850 0x7f31fc197ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f31ec02c0b0 tx=0x7f31ec032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.350+0000 7f31f37fe640 1 -- 172.21.15.92:0/3861688005 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec041020 con 0x7f31fc101850 2024-03-19T01:52:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.350+0000 7f31f37fe640 1 -- 172.21.15.92:0/3861688005 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ec02c9f0 con 0x7f31fc101850 2024-03-19T01:52:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.350+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc19fa50 con 0x7f31fc101850 2024-03-19T01:52:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.350+0000 7f31f37fe640 1 -- 172.21.15.92:0/3861688005 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec039040 con 0x7f31fc101850 2024-03-19T01:52:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.350+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc19ff40 con 0x7f31fc101850 2024-03-19T01:52:18.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.351+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31fc108bf0 con 0x7f31fc101850 2024-03-19T01:52:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.353+0000 7f31f37fe640 1 -- 172.21.15.92:0/3861688005 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f31ec044020 con 0x7f31fc101850 2024-03-19T01:52:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.353+0000 7f31f37fe640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31e80a2800 0x7f31e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.353+0000 7f31f3fff640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31e80a2800 0x7f31e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.354+0000 7f31f37fe640 1 -- 172.21.15.92:0/3861688005 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f31ec0f0630 con 0x7f31fc101850 2024-03-19T01:52:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.354+0000 7f31f3fff640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31e80a2800 0x7f31e80a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f31fc1990b0 tx=0x7f31e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:18.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.356+0000 7f31f37fe640 1 -- 172.21.15.92:0/3861688005 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f31ec04b020 con 0x7f31fc101850 2024-03-19T01:52:18.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.549+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f31fc02d440 con 0x7f31fc101850 2024-03-19T01:52:18.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.549+0000 7f31f37fe640 1 -- 172.21.15.92:0/3861688005 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f31ec0bd6e0 con 0x7f31fc101850 2024-03-19T01:52:18.549 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-03-19T01:52:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31e80a2800 msgr2=0x7f31e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31e80a2800 0x7f31e80a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f31fc1990b0 tx=0x7f31e4005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31fc101850 msgr2=0x7f31fc197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31fc101850 0x7f31fc197ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f31ec02c0b0 tx=0x7f31ec032430 comp rx=0 tx=0).stop 2024-03-19T01:52:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 shutdown_connections 2024-03-19T01:52:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc103400 0x7f31fc19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31fc102a40 0x7f31fc1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f31e80a2800 0x7f31e80a4cc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 --2- 172.21.15.92:0/3861688005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31fc101850 0x7f31fc197ba0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 >> 172.21.15.92:0/3861688005 conn(0x7f31fc0fcdc0 msgr2=0x7f31fc105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 shutdown_connections 2024-03-19T01:52:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.552+0000 7f3200e31640 1 -- 172.21.15.92:0/3861688005 wait complete. 2024-03-19T01:52:18.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:52:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.672+0000 7fd9fcca1640 1 -- 172.21.15.92:0/831515929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 msgr2=0x7fd9f8101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.672+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/831515929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 0x7fd9f8101d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009900 tx=0x7fd9ec02bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.672+0000 7fd9fcca1640 1 -- 172.21.15.92:0/831515929 shutdown_connections 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.672+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/831515929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9f81022d0 0x7fd9f8108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.672+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/831515929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 0x7fd9f8101d90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.672+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/831515929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f8100720 0x7fd9f8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.672+0000 7fd9fcca1640 1 -- 172.21.15.92:0/831515929 >> 172.21.15.92:0/831515929 conn(0x7fd9f80fbcb0 msgr2=0x7fd9f80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.672+0000 7fd9fcca1640 1 -- 172.21.15.92:0/831515929 shutdown_connections 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.672+0000 7fd9fcca1640 1 -- 172.21.15.92:0/831515929 wait complete. 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9fcca1640 1 Processor -- start 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9fcca1640 1 -- start start 2024-03-19T01:52:18.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9fcca1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9f8100720 0x7fd9f8104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9fcca1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 0x7fd9f8102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9fcca1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f81022d0 0x7fd9f8103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9fcca1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810c3a0 con 0x7fd9f8101910 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9fcca1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810c220 con 0x7fd9f81022d0 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9fcca1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810c520 con 0x7fd9f8100720 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 0x7fd9f8102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 0x7fd9f8102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36842/0 (socket says 172.21.15.92:36842) 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f5d9b640 1 -- 172.21.15.92:0/1855152855 learned_addr learned my addr 172.21.15.92:0/1855152855 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f659c640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9f8100720 0x7fd9f8104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f6d9d640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f81022d0 0x7fd9f8103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f5d9b640 1 -- 172.21.15.92:0/1855152855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9f8100720 msgr2=0x7fd9f8104640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f5d9b640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9f8100720 0x7fd9f8104640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f5d9b640 1 -- 172.21.15.92:0/1855152855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f81022d0 msgr2=0x7fd9f8103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f5d9b640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f81022d0 0x7fd9f8103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f5d9b640 1 -- 172.21.15.92:0/1855152855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f8103a50 con 0x7fd9f8101910 2024-03-19T01:52:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.673+0000 7fd9f6d9d640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f81022d0 0x7fd9f8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.674+0000 7fd9f5d9b640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 0x7fd9f8102cf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009230 tx=0x7fd9ec02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.674+0000 7fd9e77fe640 1 -- 172.21.15.92:0/1855152855 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec041070 con 0x7fd9f8101910 2024-03-19T01:52:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.674+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f81a1c00 con 0x7fd9f8101910 2024-03-19T01:52:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.674+0000 7fd9e77fe640 1 -- 172.21.15.92:0/1855152855 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ec004450 con 0x7fd9f8101910 2024-03-19T01:52:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.674+0000 7fd9e77fe640 1 -- 172.21.15.92:0/1855152855 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec003e20 con 0x7fd9f8101910 2024-03-19T01:52:18.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.674+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f81a2140 con 0x7fd9f8101910 2024-03-19T01:52:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.675+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f81a23d0 con 0x7fd9f8101910 2024-03-19T01:52:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.676+0000 7fd9e77fe640 1 -- 172.21.15.92:0/1855152855 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd9ec044070 con 0x7fd9f8101910 2024-03-19T01:52:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.676+0000 7fd9e77fe640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd9e00a2800 0x7fd9e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.677+0000 7fd9f659c640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd9e00a2800 0x7fd9e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.677+0000 7fd9e77fe640 1 -- 172.21.15.92:0/1855152855 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd9ec0f0a00 con 0x7fd9f8101910 2024-03-19T01:52:18.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.677+0000 7fd9f659c640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd9e00a2800 0x7fd9e00a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8101770 tx=0x7fd9e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:18.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.681+0000 7fd9e77fe640 1 -- 172.21.15.92:0/1855152855 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd9ec0bdab0 con 0x7fd9f8101910 2024-03-19T01:52:18.861 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.861+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd9f81a25a0 con 0x7fd9f8101910 2024-03-19T01:52:18.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.861+0000 7fd9e77fe640 1 -- 172.21.15.92:0/1855152855 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fd9ec0bd3b0 con 0x7fd9f8101910 2024-03-19T01:52:18.862 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-03-19T01:52:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.863+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd9e00a2800 msgr2=0x7fd9e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.863+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd9e00a2800 0x7fd9e00a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8101770 tx=0x7fd9e8005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.863+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 msgr2=0x7fd9f8102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.863+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 0x7fd9f8102cf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009230 tx=0x7fd9ec02c650 comp rx=0 tx=0).stop 2024-03-19T01:52:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.863+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 shutdown_connections 2024-03-19T01:52:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.863+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f81022d0 0x7fd9f8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.863+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f8101910 0x7fd9f8102cf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.864+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd9e00a2800 0x7fd9e00a4cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.864+0000 7fd9fcca1640 1 --2- 172.21.15.92:0/1855152855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9f8100720 0x7fd9f8104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.864+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 >> 172.21.15.92:0/1855152855 conn(0x7fd9f80fbcb0 msgr2=0x7fd9f80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.864+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 shutdown_connections 2024-03-19T01:52:18.865 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.864+0000 7fd9fcca1640 1 -- 172.21.15.92:0/1855152855 wait complete. 2024-03-19T01:52:18.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:52:18.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.979+0000 7f22f637e640 1 -- 172.21.15.92:0/4262566644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22f01009b0 msgr2=0x7f22f0100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.979+0000 7f22f637e640 1 --2- 172.21.15.92:0/4262566644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22f01009b0 0x7f22f0100e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009890 tx=0x7f22e402bb70 comp rx=0 tx=0).stop 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.979+0000 7f22f637e640 1 -- 172.21.15.92:0/4262566644 shutdown_connections 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 --2- 172.21.15.92:0/4262566644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f0101370 0x7f22f0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 --2- 172.21.15.92:0/4262566644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22f01009b0 0x7f22f0100e30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 --2- 172.21.15.92:0/4262566644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22f0101d10 0x7f22f0102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 -- 172.21.15.92:0/4262566644 >> 172.21.15.92:0/4262566644 conn(0x7f22f00fb420 msgr2=0x7f22f00fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 -- 172.21.15.92:0/4262566644 shutdown_connections 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 -- 172.21.15.92:0/4262566644 wait complete. 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 Processor -- start 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 -- start start 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f01009b0 0x7f22f0196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22f0101370 0x7f22f0196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22f0101d10 0x7f22f019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22f010c3d0 con 0x7f22f01009b0 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22f010c250 con 0x7f22f0101d10 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.980+0000 7f22f637e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22f010c550 con 0x7f22f0101370 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22effff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f01009b0 0x7f22f0196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22effff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f01009b0 0x7f22f0196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36860/0 (socket says 172.21.15.92:36860) 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22effff640 1 -- 172.21.15.92:0/2614846615 learned_addr learned my addr 172.21.15.92:0/2614846615 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22f0101370 0x7f22f0196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22f491b640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22f0101d10 0x7f22f019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22effff640 1 -- 172.21.15.92:0/2614846615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22f0101370 msgr2=0x7f22f0196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22effff640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22f0101370 0x7f22f0196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22effff640 1 -- 172.21.15.92:0/2614846615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22f0101d10 msgr2=0x7f22f019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22effff640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22f0101d10 0x7f22f019e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22effff640 1 -- 172.21.15.92:0/2614846615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f019e690 con 0x7f22f01009b0 2024-03-19T01:52:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22ef7fe640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22f0101370 0x7f22f0196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22effff640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f01009b0 0x7f22f0196a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f22e000e9c0 tx=0x7f22e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22ed7fa640 1 -- 172.21.15.92:0/2614846615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e000cd80 con 0x7f22f01009b0 2024-03-19T01:52:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f019e980 con 0x7f22f01009b0 2024-03-19T01:52:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22ed7fa640 1 -- 172.21.15.92:0/2614846615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e00035f0 con 0x7f22f01009b0 2024-03-19T01:52:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22ed7fa640 1 -- 172.21.15.92:0/2614846615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0010690 con 0x7f22f01009b0 2024-03-19T01:52:18.986 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.981+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f019eea0 con 0x7f22f01009b0 2024-03-19T01:52:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.982+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f0067d80 con 0x7f22f01009b0 2024-03-19T01:52:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.984+0000 7f22ed7fa640 1 -- 172.21.15.92:0/2614846615 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f22e00037a0 con 0x7f22f01009b0 2024-03-19T01:52:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.985+0000 7f22ed7fa640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22d80a2800 0x7f22d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.985+0000 7f22ef7fe640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22d80a2800 0x7f22d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.985+0000 7f22ed7fa640 1 -- 172.21.15.92:0/2614846615 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f22e00cedf0 con 0x7f22f01009b0 2024-03-19T01:52:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.985+0000 7f22ef7fe640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22d80a2800 0x7f22d80a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f22e40097b0 tx=0x7f22e402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:18.989+0000 7f22ed7fa640 1 -- 172.21.15.92:0/2614846615 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f22e009bea0 con 0x7f22f01009b0 2024-03-19T01:52:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:18 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3991274673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:18 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2169034056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:18 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3861688005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:19.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:18 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3991274673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:19.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:18 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2169034056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:19.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:18 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3861688005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:19.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.236+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22f0107b70 con 0x7f22f01009b0 2024-03-19T01:52:19.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.237+0000 7f22ed7fa640 1 -- 172.21.15.92:0/2614846615 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11932 (secure 0 0 0) 0x7f22f0107b70 con 0x7f22f01009b0 2024-03-19T01:52:19.237 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:19.237 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:50:49.881618+0000","modified":"2024-03-19T01:50:56.972139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24476,"mds_2":24475,"mds_3":24457,"mds_4":14601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6839/380126062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":380126062},{"type":"v1","addr":"172.21.15.144:6839","nonce":380126062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24476,24493,24475,24457,14601]},"id":1}]} 2024-03-19T01:52:19.238 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2024-03-19T01:52:19.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22d80a2800 msgr2=0x7f22d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22d80a2800 0x7f22d80a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f22e40097b0 tx=0x7f22e402e040 comp rx=0 tx=0).stop 2024-03-19T01:52:19.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f01009b0 msgr2=0x7f22f0196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f01009b0 0x7f22f0196a70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f22e000e9c0 tx=0x7f22e000ee80 comp rx=0 tx=0).stop 2024-03-19T01:52:19.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 shutdown_connections 2024-03-19T01:52:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22f0101d10 0x7f22f019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f22d80a2800 0x7f22d80a4cc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22f0101370 0x7f22f0196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 --2- 172.21.15.92:0/2614846615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f01009b0 0x7f22f0196a70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 >> 172.21.15.92:0/2614846615 conn(0x7f22f00fb420 msgr2=0x7f22f0104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 shutdown_connections 2024-03-19T01:52:19.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.239+0000 7f22f637e640 1 -- 172.21.15.92:0/2614846615 wait complete. 2024-03-19T01:52:19.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:18 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3991274673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:19.249 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:18 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2169034056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:19.250 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:18 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3861688005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:19.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:52:19.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 -- 172.21.15.92:0/1067439454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86c01022f0 msgr2=0x7f86c0108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 --2- 172.21.15.92:0/1067439454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86c01022f0 0x7f86c0108b80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009900 tx=0x7f86b002bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:19.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 -- 172.21.15.92:0/1067439454 shutdown_connections 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 --2- 172.21.15.92:0/1067439454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86c01022f0 0x7f86c0108b80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 --2- 172.21.15.92:0/1067439454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0101930 0x7f86c0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 --2- 172.21.15.92:0/1067439454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c0100740 0x7f86c0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 -- 172.21.15.92:0/1067439454 >> 172.21.15.92:0/1067439454 conn(0x7f86c00fbcb0 msgr2=0x7f86c00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 -- 172.21.15.92:0/1067439454 shutdown_connections 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 -- 172.21.15.92:0/1067439454 wait complete. 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.355+0000 7f86c55df640 1 Processor -- start 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86c55df640 1 -- start start 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86c55df640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0100740 0x7f86c0069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86c55df640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86c0101930 0x7f86c019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86c55df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c01022f0 0x7f86c019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86c55df640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010c3a0 con 0x7f86c0100740 2024-03-19T01:52:19.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86c55df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010c220 con 0x7f86c01022f0 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86c55df640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010c520 con 0x7f86c0101930 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86beffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0100740 0x7f86c0069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86beffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0100740 0x7f86c0069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36890/0 (socket says 172.21.15.92:36890) 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86beffd640 1 -- 172.21.15.92:0/4262509883 learned_addr learned my addr 172.21.15.92:0/4262509883 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86be7fc640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86c0101930 0x7f86c019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86bf7fe640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c01022f0 0x7f86c019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86beffd640 1 -- 172.21.15.92:0/4262509883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86c0101930 msgr2=0x7f86c019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86beffd640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86c0101930 0x7f86c019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86beffd640 1 -- 172.21.15.92:0/4262509883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c01022f0 msgr2=0x7f86c019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86beffd640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c01022f0 0x7f86c019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86beffd640 1 -- 172.21.15.92:0/4262509883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c019f6c0 con 0x7f86c0100740 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86be7fc640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86c0101930 0x7f86c019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86beffd640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0100740 0x7f86c0069510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86b800e9c0 tx=0x7f86b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:19.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.356+0000 7f86a7fff640 1 -- 172.21.15.92:0/4262509883 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b800cd80 con 0x7f86c0100740 2024-03-19T01:52:19.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.357+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c019f9b0 con 0x7f86c0100740 2024-03-19T01:52:19.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.357+0000 7f86a7fff640 1 -- 172.21.15.92:0/4262509883 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b80035f0 con 0x7f86c0100740 2024-03-19T01:52:19.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.357+0000 7f86a7fff640 1 -- 172.21.15.92:0/4262509883 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8010690 con 0x7f86c0100740 2024-03-19T01:52:19.359 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.357+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c019ff50 con 0x7f86c0100740 2024-03-19T01:52:19.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.358+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c0068610 con 0x7f86c0100740 2024-03-19T01:52:19.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.359+0000 7f86a7fff640 1 -- 172.21.15.92:0/4262509883 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f86b80037a0 con 0x7f86c0100740 2024-03-19T01:52:19.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.360+0000 7f86a7fff640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f86a80a2800 0x7f86a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.360+0000 7f86be7fc640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f86a80a2800 0x7f86a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.360+0000 7f86a7fff640 1 -- 172.21.15.92:0/4262509883 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f86b80cee00 con 0x7f86c0100740 2024-03-19T01:52:19.363 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.360+0000 7f86be7fc640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f86a80a2800 0x7f86a80a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f86c0069280 tx=0x7f86b4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:19.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.365+0000 7f86a7fff640 1 -- 172.21.15.92:0/4262509883 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f86b809beb0 con 0x7f86c0100740 2024-03-19T01:52:19.544 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.544+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86c0062420 con 0x7f86c0100740 2024-03-19T01:52:19.546 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.547+0000 7f86a7fff640 1 -- 172.21.15.92:0/4262509883 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22690 (secure 0 0 0) 0x7f86b809b7b0 con 0x7f86c0100740 2024-03-19T01:52:19.546 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:19.546 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":87,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:51:40.581546+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:52:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f86a80a2800 msgr2=0x7f86a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f86a80a2800 0x7f86a80a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f86c0069280 tx=0x7f86b4009500 comp rx=0 tx=0).stop 2024-03-19T01:52:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0100740 msgr2=0x7f86c0069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0100740 0x7f86c0069510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86b800e9c0 tx=0x7f86b800ee80 comp rx=0 tx=0).stop 2024-03-19T01:52:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 shutdown_connections 2024-03-19T01:52:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c01022f0 0x7f86c019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f86a80a2800 0x7f86a80a4cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86c0101930 0x7f86c019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 --2- 172.21.15.92:0/4262509883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86c0100740 0x7f86c0069510 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 >> 172.21.15.92:0/4262509883 conn(0x7f86c00fbcb0 msgr2=0x7f86c0104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 shutdown_connections 2024-03-19T01:52:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.549+0000 7f86c55df640 1 -- 172.21.15.92:0/4262509883 wait complete. 2024-03-19T01:52:19.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:46:07.695753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:50:47.888441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:50:49.068308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:52:19.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:52:19.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 -- 172.21.15.92:0/417187592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 msgr2=0x7fef041098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 --2- 172.21.15.92:0/417187592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 0x7fef041098c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7feef4009900 tx=0x7feef402bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:19.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 -- 172.21.15.92:0/417187592 shutdown_connections 2024-03-19T01:52:19.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 --2- 172.21.15.92:0/417187592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 0x7fef041098c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 --2- 172.21.15.92:0/417187592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04104ba0 0x7fef04106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 --2- 172.21.15.92:0/417187592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef04102270 0x7fef04104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 -- 172.21.15.92:0/417187592 >> 172.21.15.92:0/417187592 conn(0x7fef040fbbc0 msgr2=0x7fef040fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:19.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 -- 172.21.15.92:0/417187592 shutdown_connections 2024-03-19T01:52:19.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 -- 172.21.15.92:0/417187592 wait complete. 2024-03-19T01:52:19.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.666+0000 7fef09801640 1 Processor -- start 2024-03-19T01:52:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef09801640 1 -- start start 2024-03-19T01:52:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef09801640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef04102270 0x7fef04197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef09801640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04104ba0 0x7fef04197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef09801640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 0x7fef0419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef09801640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410d3e0 con 0x7fef041074d0 2024-03-19T01:52:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef09801640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410d260 con 0x7fef04104ba0 2024-03-19T01:52:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef09801640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef0410d560 con 0x7fef04102270 2024-03-19T01:52:19.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef037fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 0x7fef0419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef037fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 0x7fef0419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36898/0 (socket says 172.21.15.92:36898) 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef037fe640 1 -- 172.21.15.92:0/3392364482 learned_addr learned my addr 172.21.15.92:0/3392364482 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef02ffd640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef04102270 0x7fef04197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef027fc640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04104ba0 0x7fef04197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef037fe640 1 -- 172.21.15.92:0/3392364482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef04102270 msgr2=0x7fef04197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef037fe640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef04102270 0x7fef04197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef037fe640 1 -- 172.21.15.92:0/3392364482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04104ba0 msgr2=0x7fef04197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef037fe640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04104ba0 0x7fef04197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef037fe640 1 -- 172.21.15.92:0/3392364482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0419f620 con 0x7fef041074d0 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef037fe640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 0x7fef0419efb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7feef4005ab0 tx=0x7feef402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7feeebfff640 1 -- 172.21.15.92:0/3392364482 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4041070 con 0x7fef041074d0 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0419f8b0 con 0x7fef041074d0 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.667+0000 7feeebfff640 1 -- 172.21.15.92:0/3392364482 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef4004450 con 0x7fef041074d0 2024-03-19T01:52:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.668+0000 7feeebfff640 1 -- 172.21.15.92:0/3392364482 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4003e60 con 0x7fef041074d0 2024-03-19T01:52:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.668+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0419fda0 con 0x7fef041074d0 2024-03-19T01:52:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.669+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef040fd6b0 con 0x7fef041074d0 2024-03-19T01:52:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.669+0000 7feeebfff640 1 -- 172.21.15.92:0/3392364482 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7feef4044070 con 0x7fef041074d0 2024-03-19T01:52:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.670+0000 7feeebfff640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7feeec0a2800 0x7feeec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.670+0000 7fef02ffd640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7feeec0a2800 0x7feeec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.670+0000 7feeebfff640 1 -- 172.21.15.92:0/3392364482 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7feef40f0670 con 0x7fef041074d0 2024-03-19T01:52:19.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.670+0000 7fef02ffd640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7feeec0a2800 0x7feeec0a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feefc005fd0 tx=0x7feefc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.677+0000 7feeebfff640 1 -- 172.21.15.92:0/3392364482 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feef40bd0d0 con 0x7fef041074d0 2024-03-19T01:52:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.864+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef04062420 con 0x7fef041074d0 2024-03-19T01:52:19.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.865+0000 7feeebfff640 1 -- 172.21.15.92:0/3392364482 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7feef40c1170 con 0x7fef041074d0 2024-03-19T01:52:19.864 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-03-19T01:52:19.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7feeec0a2800 msgr2=0x7feeec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7feeec0a2800 0x7feeec0a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feefc005fd0 tx=0x7feefc009500 comp rx=0 tx=0).stop 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 msgr2=0x7fef0419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 0x7fef0419efb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7feef4005ab0 tx=0x7feef402c6c0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 shutdown_connections 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef041074d0 0x7fef0419efb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef04104ba0 0x7fef04197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7feeec0a2800 0x7feeec0a4cc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 --2- 172.21.15.92:0/3392364482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef04102270 0x7fef04197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 >> 172.21.15.92:0/3392364482 conn(0x7fef040fbbc0 msgr2=0x7fef04102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 shutdown_connections 2024-03-19T01:52:19.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.867+0000 7fef09801640 1 -- 172.21.15.92:0/3392364482 wait complete. 2024-03-19T01:52:19.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:52:19.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/1772901146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e01022d0 msgr2=0x7fd3e0108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/1772901146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e01022d0 0x7fd3e0108b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009ca0 tx=0x7fd3d802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/1772901146 shutdown_connections 2024-03-19T01:52:19.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/1772901146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e01022d0 0x7fd3e0108b60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/1772901146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3e0101910 0x7fd3e0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.989 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/1772901146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e0100720 0x7fd3e0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/1772901146 >> 172.21.15.92:0/1772901146 conn(0x7fd3e00fbcb0 msgr2=0x7fd3e00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/1772901146 shutdown_connections 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/1772901146 wait complete. 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 Processor -- start 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.988+0000 7fd3e5f3b640 1 -- start start 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3e5f3b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e0100720 0x7fd3e00694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3e5f3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3e0101910 0x7fd3e019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3e5f3b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e01022d0 0x7fd3e019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3e5f3b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e010c380 con 0x7fd3e0100720 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3e5f3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e010c200 con 0x7fd3e0101910 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3e5f3b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e010c500 con 0x7fd3e01022d0 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e0100720 0x7fd3e00694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.990 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e0100720 0x7fd3e00694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36910/0 (socket says 172.21.15.92:36910) 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3df7fe640 1 -- 172.21.15.92:0/3909288806 learned_addr learned my addr 172.21.15.92:0/3909288806 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3deffd640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3e0101910 0x7fd3e019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3dffff640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e01022d0 0x7fd3e019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3df7fe640 1 -- 172.21.15.92:0/3909288806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e01022d0 msgr2=0x7fd3e019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3df7fe640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e01022d0 0x7fd3e019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3df7fe640 1 -- 172.21.15.92:0/3909288806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3e0101910 msgr2=0x7fd3e019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3df7fe640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3e0101910 0x7fd3e019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3df7fe640 1 -- 172.21.15.92:0/3909288806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e019f770 con 0x7fd3e0100720 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3df7fe640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e0100720 0x7fd3e00694f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000e9c0 tx=0x7fd3d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3dcff9640 1 -- 172.21.15.92:0/3909288806 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000cd80 con 0x7fd3e0100720 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e019fa00 con 0x7fd3e0100720 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3dcff9640 1 -- 172.21.15.92:0/3909288806 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d00035f0 con 0x7fd3e0100720 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3dcff9640 1 -- 172.21.15.92:0/3909288806 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0010690 con 0x7fd3e0100720 2024-03-19T01:52:19.991 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.989+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e019ffa0 con 0x7fd3e0100720 2024-03-19T01:52:19.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.990+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e0068610 con 0x7fd3e0100720 2024-03-19T01:52:19.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.992+0000 7fd3dcff9640 1 -- 172.21.15.92:0/3909288806 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd3d00037a0 con 0x7fd3e0100720 2024-03-19T01:52:19.992 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.992+0000 7fd3dcff9640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd3c80a2800 0x7fd3c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:19.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.992+0000 7fd3deffd640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd3c80a2800 0x7fd3c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:19.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.993+0000 7fd3dcff9640 1 -- 172.21.15.92:0/3909288806 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd3d00ceee0 con 0x7fd3e0100720 2024-03-19T01:52:19.993 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.993+0000 7fd3deffd640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd3c80a2800 0x7fd3c80a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0069260 tx=0x7fd3d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:19.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:19.996+0000 7fd3dcff9640 1 -- 172.21.15.92:0/3909288806 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd3d009c020 con 0x7fd3e0100720 2024-03-19T01:52:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:19 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1855152855' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:19 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2614846615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:19 smithi144 ceph-mon[67199]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-19T01:52:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:19 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4262509883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:20.106 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:19 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1855152855' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:20.106 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:19 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2614846615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:20.106 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:19 smithi159 ceph-mon[67110]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-19T01:52:20.106 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:19 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4262509883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.191+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd3e002d270 con 0x7fd3e0100720 2024-03-19T01:52:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.192+0000 7fd3dcff9640 1 -- 172.21.15.92:0/3909288806 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fd3d009b920 con 0x7fd3e0100720 2024-03-19T01:52:20.191 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-03-19T01:52:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.193+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd3c80a2800 msgr2=0x7fd3c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.193+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd3c80a2800 0x7fd3c80a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0069260 tx=0x7fd3d4005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.193+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e0100720 msgr2=0x7fd3e00694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.193+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e0100720 0x7fd3e00694f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000e9c0 tx=0x7fd3d000ee80 comp rx=0 tx=0).stop 2024-03-19T01:52:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.194+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 shutdown_connections 2024-03-19T01:52:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.194+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd3c80a2800 0x7fd3c80a4cc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.194+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e01022d0 0x7fd3e019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.194+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3e0101910 0x7fd3e019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.194+0000 7fd3e5f3b640 1 --2- 172.21.15.92:0/3909288806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3e0100720 0x7fd3e00694f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.194+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 >> 172.21.15.92:0/3909288806 conn(0x7fd3e00fbcb0 msgr2=0x7fd3e0104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:20.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.194+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 shutdown_connections 2024-03-19T01:52:20.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.194+0000 7fd3e5f3b640 1 -- 172.21.15.92:0/3909288806 wait complete. 2024-03-19T01:52:20.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:19 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1855152855' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:20.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:19 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2614846615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:20.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:19 smithi092 ceph-mon[62574]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-19T01:52:20.203 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:19 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4262509883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:20.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:52:20.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 -- 172.21.15.92:0/484254124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8102ad0 msgr2=0x7f07b8102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/484254124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8102ad0 0x7f07b8102f50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009930 tx=0x7f07ac02bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 -- 172.21.15.92:0/484254124 shutdown_connections 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/484254124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b8103490 0x7f07b8109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/484254124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8102ad0 0x7f07b8102f50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/484254124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b80ffb00 0x7f07b80fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 -- 172.21.15.92:0/484254124 >> 172.21.15.92:0/484254124 conn(0x7f07b80fb430 msgr2=0x7f07b80fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 -- 172.21.15.92:0/484254124 shutdown_connections 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 -- 172.21.15.92:0/484254124 wait complete. 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.310+0000 7f07bfbc6640 1 Processor -- start 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bfbc6640 1 -- start start 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bfbc6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b80ffb00 0x7f07b8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bfbc6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102ad0 0x7f07b81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bfbc6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8103490 0x7f07b819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bfbc6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f07b810d4c0 con 0x7f07b8103490 2024-03-19T01:52:20.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bfbc6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07b810d340 con 0x7f07b80ffb00 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bfbc6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07b810d640 con 0x7f07b8102ad0 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07be163640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8103490 0x7f07b819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07be163640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8103490 0x7f07b819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36924/0 (socket says 172.21.15.92:36924) 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07be163640 1 -- 172.21.15.92:0/1312685815 learned_addr learned my addr 172.21.15.92:0/1312685815 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bd962640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b80ffb00 0x7f07b8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bd161640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102ad0 0x7f07b81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07be163640 1 -- 172.21.15.92:0/1312685815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102ad0 msgr2=0x7f07b81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07be163640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102ad0 0x7f07b81980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07be163640 1 -- 172.21.15.92:0/1312685815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b80ffb00 msgr2=0x7f07b8197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07be163640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b80ffb00 0x7f07b8197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07be163640 1 -- 172.21.15.92:0/1312685815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b819f780 con 0x7f07b8103490 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07be163640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8103490 0x7f07b819f110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f07a800eb10 tx=0x7f07a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07b6ffd640 1 -- 172.21.15.92:0/1312685815 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8004b20 con 0x7f07b8103490 2024-03-19T01:52:20.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07b6ffd640 1 -- 172.21.15.92:0/1312685815 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07a8004cc0 con 0x7f07b8103490 2024-03-19T01:52:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b819fa70 con 0x7f07b8103490 2024-03-19T01:52:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07b6ffd640 1 -- 172.21.15.92:0/1312685815 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8010760 con 0x7f07b8103490 2024-03-19T01:52:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.311+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b819ff90 con 0x7f07b8103490 2024-03-19T01:52:20.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.312+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b8108c00 con 0x7f07b8103490 2024-03-19T01:52:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.313+0000 7f07b6ffd640 1 -- 172.21.15.92:0/1312685815 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f07a8005020 con 0x7f07b8103490 2024-03-19T01:52:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.314+0000 7f07b6ffd640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07a00a2880 0x7f07a00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.314+0000 7f07bd962640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07a00a2880 0x7f07a00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.314+0000 7f07b6ffd640 1 -- 172.21.15.92:0/1312685815 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f07a80cef30 con 0x7f07b8103490 2024-03-19T01:52:20.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.315+0000 7f07bd962640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07a00a2880 0x7f07a00a4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f07b00095f0 tx=0x7f07b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:20.318 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.319+0000 7f07b6ffd640 1 -- 172.21.15.92:0/1312685815 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f07a809bfe0 con 0x7f07b8103490 2024-03-19T01:52:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.534+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f07b802d240 con 0x7f07b8103490 2024-03-19T01:52:20.533 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.534+0000 7f07b6ffd640 1 -- 172.21.15.92:0/1312685815 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f07a809b8e0 con 0x7f07b8103490 2024-03-19T01:52:20.534 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07a00a2880 msgr2=0x7f07a00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07a00a2880 0x7f07a00a4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f07b00095f0 tx=0x7f07b0005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8103490 msgr2=0x7f07b819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8103490 0x7f07b819f110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f07a800eb10 tx=0x7f07a800c2d0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 shutdown_connections 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07b8103490 0x7f07b819f110 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f07a00a2880 0x7f07a00a4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102ad0 0x7f07b81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 --2- 172.21.15.92:0/1312685815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07b80ffb00 0x7f07b8197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 >> 172.21.15.92:0/1312685815 conn(0x7f07b80fb430 msgr2=0x7f07b8105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 shutdown_connections 2024-03-19T01:52:20.536 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.536+0000 7f07bfbc6640 1 -- 172.21.15.92:0/1312685815 wait complete. 2024-03-19T01:52:20.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:52:20.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.651+0000 7fd02d201640 1 -- 172.21.15.92:0/1297875948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0281017a0 msgr2=0x7fd028101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.651+0000 7fd02d201640 1 --2- 172.21.15.92:0/1297875948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0281017a0 0x7fd028101c20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd010009900 tx=0x7fd01002bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:20.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.651+0000 7fd02d201640 1 -- 172.21.15.92:0/1297875948 shutdown_connections 2024-03-19T01:52:20.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.651+0000 7fd02d201640 1 --2- 172.21.15.92:0/1297875948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd028102160 0x7fd0281089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.651+0000 7fd02d201640 1 --2- 172.21.15.92:0/1297875948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0281017a0 0x7fd028101c20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.651+0000 7fd02d201640 1 --2- 172.21.15.92:0/1297875948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0281005b0 0x7fd0281009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.651+0000 7fd02d201640 1 -- 172.21.15.92:0/1297875948 >> 172.21.15.92:0/1297875948 conn(0x7fd0280fbb00 msgr2=0x7fd0280fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:20.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.651+0000 7fd02d201640 1 -- 172.21.15.92:0/1297875948 shutdown_connections 2024-03-19T01:52:20.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.651+0000 7fd02d201640 1 -- 172.21.15.92:0/1297875948 wait complete. 2024-03-19T01:52:20.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.652+0000 7fd02d201640 1 Processor -- start 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.652+0000 7fd02d201640 1 -- start start 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02d201640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0281005b0 0x7fd028196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02d201640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0281017a0 0x7fd028196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02d201640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd028102160 0x7fd02819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02d201640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd02810c210 con 0x7fd028102160 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02d201640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd02810c090 con 0x7fd0281017a0 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02d201640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd02810c390 con 0x7fd0281005b0 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd028102160 0x7fd02819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd028102160 0x7fd02819de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36938/0 (socket says 172.21.15.92:36938) 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02759e640 1 -- 172.21.15.92:0/1220816400 learned_addr learned my addr 172.21.15.92:0/1220816400 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd026d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0281005b0 0x7fd028196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02659c640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0281017a0 0x7fd028196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02759e640 1 -- 172.21.15.92:0/1220816400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0281005b0 msgr2=0x7fd028196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02759e640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0281005b0 0x7fd028196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02759e640 1 -- 172.21.15.92:0/1220816400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0281017a0 msgr2=0x7fd028196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02759e640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0281017a0 0x7fd028196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02759e640 1 -- 172.21.15.92:0/1220816400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02819e4a0 con 0x7fd028102160 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd026d9d640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0281005b0 0x7fd028196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02759e640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd028102160 0x7fd02819de30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd01800eb00 tx=0x7fd01800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd01ffff640 1 -- 172.21.15.92:0/1220816400 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01800bbb0 con 0x7fd028102160 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0280689e0 con 0x7fd028102160 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd01ffff640 1 -- 172.21.15.92:0/1220816400 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01800bd50 con 0x7fd028102160 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd01ffff640 1 -- 172.21.15.92:0/1220816400 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd018017410 con 0x7fd028102160 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.653+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd028068f00 con 0x7fd028102160 2024-03-19T01:52:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.654+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd028068530 con 0x7fd028102160 2024-03-19T01:52:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.655+0000 7fd01ffff640 1 -- 172.21.15.92:0/1220816400 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd018004fa0 con 0x7fd028102160 2024-03-19T01:52:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.656+0000 7fd01ffff640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0140a2800 0x7fd0140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.656+0000 7fd026d9d640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0140a2800 0x7fd0140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.656+0000 7fd01ffff640 1 -- 172.21.15.92:0/1220816400 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd0180d01e0 con 0x7fd028102160 2024-03-19T01:52:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.657+0000 7fd026d9d640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0140a2800 0x7fd0140a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd028101600 tx=0x7fd020009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:20.659 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.660+0000 7fd01ffff640 1 -- 172.21.15.92:0/1220816400 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd01809cc40 con 0x7fd028102160 2024-03-19T01:52:20.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.929+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd02819e670 con 0x7fd028102160 2024-03-19T01:52:20.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.931+0000 7fd01ffff640 1 -- 172.21.15.92:0/1220816400 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11932 (secure 0 0 0) 0x7fd0180a0ce0 con 0x7fd028102160 2024-03-19T01:52:20.930 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:20.930 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:50:49.881618+0000","modified":"2024-03-19T01:50:56.972139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24476,"mds_2":24475,"mds_3":24457,"mds_4":14601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6839/380126062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":380126062},{"type":"v1","addr":"172.21.15.144:6839","nonce":380126062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24476,24493,24475,24457,14601]},"id":1}]} 2024-03-19T01:52:20.930 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2024-03-19T01:52:20.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.932+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0140a2800 msgr2=0x7fd0140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.932+0000 7fd02d201640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0140a2800 0x7fd0140a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd028101600 tx=0x7fd020009500 comp rx=0 tx=0).stop 2024-03-19T01:52:20.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.932+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd028102160 msgr2=0x7fd02819de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:20.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.932+0000 7fd02d201640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd028102160 0x7fd02819de30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd01800eb00 tx=0x7fd01800ef00 comp rx=0 tx=0).stop 2024-03-19T01:52:20.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.933+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 shutdown_connections 2024-03-19T01:52:20.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.933+0000 7fd02d201640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd028102160 0x7fd02819de30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.932 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.933+0000 7fd02d201640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0281017a0 0x7fd028196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.933+0000 7fd02d201640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0140a2800 0x7fd0140a4cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.933+0000 7fd02d201640 1 --2- 172.21.15.92:0/1220816400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0281005b0 0x7fd028196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.933+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 >> 172.21.15.92:0/1220816400 conn(0x7fd0280fbb00 msgr2=0x7fd028104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.933+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 shutdown_connections 2024-03-19T01:52:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:20.933+0000 7fd02d201640 1 -- 172.21.15.92:0/1220816400 wait complete. 2024-03-19T01:52:20.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.059+0000 7ff8868c0640 1 -- 172.21.15.92:0/1319856660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 msgr2=0x7ff880100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.059+0000 7ff8868c0640 1 --2- 172.21.15.92:0/1319856660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 0x7ff880100b60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff870009900 tx=0x7ff87002bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.059+0000 7ff8868c0640 1 -- 172.21.15.92:0/1319856660 shutdown_connections 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.059+0000 7ff8868c0640 1 --2- 172.21.15.92:0/1319856660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8801022f0 0x7ff880108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.059+0000 7ff8868c0640 1 --2- 172.21.15.92:0/1319856660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff880101930 0x7ff880101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 --2- 172.21.15.92:0/1319856660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 0x7ff880100b60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 -- 172.21.15.92:0/1319856660 >> 172.21.15.92:0/1319856660 conn(0x7ff8800fbcb0 msgr2=0x7ff8800fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 -- 172.21.15.92:0/1319856660 shutdown_connections 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 -- 172.21.15.92:0/1319856660 wait complete. 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 Processor -- start 2024-03-19T01:52:21.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 -- start start 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 0x7ff880196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff880101930 0x7ff880196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8801022f0 0x7ff88019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff88010c3c0 con 0x7ff880100740 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff88010c240 con 0x7ff8801022f0 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.060+0000 7ff8868c0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff88010c540 con 0x7ff880101930 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 0x7ff880196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 0x7ff880196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36950/0 (socket says 172.21.15.92:36950) 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87ffff640 1 -- 172.21.15.92:0/2578226651 learned_addr learned my addr 172.21.15.92:0/2578226651 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87f7fe640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff880101930 0x7ff880196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff884e5d640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8801022f0 0x7ff88019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87ffff640 1 -- 172.21.15.92:0/2578226651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff880101930 msgr2=0x7ff880196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87ffff640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff880101930 0x7ff880196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87ffff640 1 -- 172.21.15.92:0/2578226651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8801022f0 msgr2=0x7ff88019dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87ffff640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8801022f0 0x7ff88019dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87ffff640 1 -- 172.21.15.92:0/2578226651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88019e650 con 0x7ff880100740 2024-03-19T01:52:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87ffff640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 0x7ff880196a30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff870000c00 tx=0x7ff87002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87d7fa640 1 -- 172.21.15.92:0/2578226651 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870004450 con 0x7ff880100740 2024-03-19T01:52:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff880068980 con 0x7ff880100740 2024-03-19T01:52:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87d7fa640 1 -- 172.21.15.92:0/2578226651 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8700054e0 con 0x7ff880100740 2024-03-19T01:52:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff87d7fa640 1 -- 172.21.15.92:0/2578226651 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87003b440 con 0x7ff880100740 2024-03-19T01:52:21.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.061+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff880068ec0 con 0x7ff880100740 2024-03-19T01:52:21.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.062+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff880068610 con 0x7ff880100740 2024-03-19T01:52:21.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.063+0000 7ff87d7fa640 1 -- 172.21.15.92:0/2578226651 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff870003780 con 0x7ff880100740 2024-03-19T01:52:21.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.064+0000 7ff87d7fa640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff8680a2800 0x7ff8680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.064+0000 7ff87f7fe640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff8680a2800 0x7ff8680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.064+0000 7ff87d7fa640 1 -- 172.21.15.92:0/2578226651 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff8700f2530 con 0x7ff880100740 2024-03-19T01:52:21.064 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.064+0000 7ff87f7fe640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff8680a2800 0x7ff8680a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff880197f40 tx=0x7ff874005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.069+0000 7ff87d7fa640 1 -- 172.21.15.92:0/2578226651 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff8700bf670 con 0x7ff880100740 2024-03-19T01:52:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:20 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3392364482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:20 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3909288806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:20 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1312685815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:21.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:20 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3392364482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:21.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:20 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3909288806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:21.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:20 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1312685815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:21.249 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.249+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff88019e8e0 con 0x7ff880100740 2024-03-19T01:52:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.252+0000 7ff87d7fa640 1 -- 172.21.15.92:0/2578226651 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22690 (secure 0 0 0) 0x7ff8700bef70 con 0x7ff880100740 2024-03-19T01:52:21.251 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:21.251 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":87,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:51:40.581546+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff8680a2800 msgr2=0x7ff8680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff8680a2800 0x7ff8680a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff880197f40 tx=0x7ff874005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 msgr2=0x7ff880196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 0x7ff880196a30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff870000c00 tx=0x7ff87002c640 comp rx=0 tx=0).stop 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 shutdown_connections 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8801022f0 0x7ff88019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff8680a2800 0x7ff8680a4cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff880101930 0x7ff880196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 --2- 172.21.15.92:0/2578226651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff880100740 0x7ff880196a30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 >> 172.21.15.92:0/2578226651 conn(0x7ff8800fbcb0 msgr2=0x7ff880104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 shutdown_connections 2024-03-19T01:52:21.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.255+0000 7ff8868c0640 1 -- 172.21.15.92:0/2578226651 wait complete. 2024-03-19T01:52:21.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:20 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3392364482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:21.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:20 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3909288806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:21.267 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:20 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1312685815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:21.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:52:21.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 -- 172.21.15.92:0/1497292942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0101930 msgr2=0x7fc4d0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/1497292942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0101930 0x7fc4d0101db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009900 tx=0x7fc4c402bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:21.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 -- 172.21.15.92:0/1497292942 shutdown_connections 2024-03-19T01:52:21.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/1497292942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d01022f0 0x7fc4d0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/1497292942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d0101930 0x7fc4d0101db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.375 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/1497292942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4d0100740 0x7fc4d0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 -- 172.21.15.92:0/1497292942 >> 172.21.15.92:0/1497292942 conn(0x7fc4d00fbcb0 msgr2=0x7fc4d00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 -- 172.21.15.92:0/1497292942 shutdown_connections 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 -- 172.21.15.92:0/1497292942 wait complete. 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 Processor -- start 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.373+0000 7fc4d7a97640 1 -- start start 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d7a97640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d0100740 0x7fc4d00694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d7a97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4d0101930 0x7fc4d019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d7a97640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d01022f0 0x7fc4d019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d7a97640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d010c3c0 con 0x7fc4d01022f0 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d7a97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d010c240 con 0x7fc4d0101930 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d7a97640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d010c540 con 0x7fc4d0100740 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d6034640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d01022f0 0x7fc4d019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d6034640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d01022f0 0x7fc4d019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36968/0 (socket says 172.21.15.92:36968) 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d6034640 1 -- 172.21.15.92:0/3277445244 learned_addr learned my addr 172.21.15.92:0/3277445244 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d5833640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d0100740 0x7fc4d00694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d5032640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4d0101930 0x7fc4d019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d6034640 1 -- 172.21.15.92:0/3277445244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d0100740 msgr2=0x7fc4d00694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d6034640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d0100740 0x7fc4d00694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d6034640 1 -- 172.21.15.92:0/3277445244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4d0101930 msgr2=0x7fc4d019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d6034640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4d0101930 0x7fc4d019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d6034640 1 -- 172.21.15.92:0/3277445244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d019f7e0 con 0x7fc4d01022f0 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d6034640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d01022f0 0x7fc4d019eed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000e970 tx=0x7fc4c000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4ceffd640 1 -- 172.21.15.92:0/3277445244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c000ccc0 con 0x7fc4d01022f0 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4ceffd640 1 -- 172.21.15.92:0/3277445244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c000ce60 con 0x7fc4d01022f0 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4ceffd640 1 -- 172.21.15.92:0/3277445244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0010490 con 0x7fc4d01022f0 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d019fa70 con 0x7fc4d01022f0 2024-03-19T01:52:21.377 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.374+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d019ff90 con 0x7fc4d01022f0 2024-03-19T01:52:21.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.375+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d0107a60 con 0x7fc4d01022f0 2024-03-19T01:52:21.378 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.377+0000 7fc4ceffd640 1 -- 172.21.15.92:0/3277445244 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc4c0005000 con 0x7fc4d01022f0 2024-03-19T01:52:21.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.378+0000 7fc4ceffd640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4b80a2800 0x7fc4b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.378+0000 7fc4d5833640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4b80a2800 0x7fc4b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.378+0000 7fc4ceffd640 1 -- 172.21.15.92:0/3277445244 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc4c00cf0e0 con 0x7fc4d01022f0 2024-03-19T01:52:21.379 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.378+0000 7fc4d5833640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4b80a2800 0x7fc4b80a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80099a0 tx=0x7fc4c8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.381+0000 7fc4ceffd640 1 -- 172.21.15.92:0/3277445244 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc4c0053740 con 0x7fc4d01022f0 2024-03-19T01:52:21.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.599+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4d002d1f0 con 0x7fc4d01022f0 2024-03-19T01:52:21.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.600+0000 7fc4ceffd640 1 -- 172.21.15.92:0/3277445244 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11932 (secure 0 0 0) 0x7fc4c009c130 con 0x7fc4d01022f0 2024-03-19T01:52:21.601 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:21.601 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:50:49.881618+0000","modified":"2024-03-19T01:50:56.972139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24476,"mds_2":24475,"mds_3":24457,"mds_4":14601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6839/380126062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":380126062},{"type":"v1","addr":"172.21.15.144:6839","nonce":380126062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24476,24493,24475,24457,14601]},"id":1}]} 2024-03-19T01:52:21.601 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4b80a2800 msgr2=0x7fc4b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4b80a2800 0x7fc4b80a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80099a0 tx=0x7fc4c8009500 comp rx=0 tx=0).stop 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d01022f0 msgr2=0x7fc4d019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d01022f0 0x7fc4d019eed0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000e970 tx=0x7fc4c000ed70 comp rx=0 tx=0).stop 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 shutdown_connections 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4d01022f0 0x7fc4d019eed0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4d0101930 0x7fc4d019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc4b80a2800 0x7fc4b80a4cc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 --2- 172.21.15.92:0/3277445244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d0100740 0x7fc4d00694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 >> 172.21.15.92:0/3277445244 conn(0x7fc4d00fbcb0 msgr2=0x7fc4d0104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.606 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 shutdown_connections 2024-03-19T01:52:21.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.602+0000 7fc4d7a97640 1 -- 172.21.15.92:0/3277445244 wait complete. 2024-03-19T01:52:21.614 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-19T01:50:49.881618+0000', 'modified': '2024-03-19T01:50:56.972139+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24493, 'mds_1': 24476, 'mds_2': 24475, 'mds_3': 24457, 'mds_4': 14601}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14589': {'gid': 14589, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6839/1427243227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1427243227}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1427243227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14601': {'gid': 14601, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6841/133114375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6840', 'nonce': 133114375}, {'type': 'v1', 'addr': '172.21.15.92:6841', 'nonce': 133114375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6833/3429474924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3429474924}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3429474924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/2148025721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 2148025721}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 2148025721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/3252974913', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 3252974913}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 3252974913}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6837/1367568659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 1367568659}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 1367568659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.159:6835/610100257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 610100257}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 610100257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.159:6837/3903346711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 3903346711}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 3903346711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.144:6839/380126062', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 380126062}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 380126062}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.159:6839/2741545135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 2741545135}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 2741545135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24493, 'qdb_cluster': [24476, 24493, 24475, 24457, 14601]} 2024-03-19T01:52:21.614 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-19T01:52:21.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-19T01:52:21.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.725+0000 7f109875b640 1 -- 172.21.15.92:0/749895278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10901022d0 msgr2=0x7f1090108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.725+0000 7f109875b640 1 --2- 172.21.15.92:0/749895278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10901022d0 0x7f1090108b60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1088009ca0 tx=0x7f108802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.725+0000 7f109875b640 1 -- 172.21.15.92:0/749895278 shutdown_connections 2024-03-19T01:52:21.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.725+0000 7f109875b640 1 --2- 172.21.15.92:0/749895278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10901022d0 0x7f1090108b60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.725+0000 7f109875b640 1 --2- 172.21.15.92:0/749895278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1090101910 0x7f1090101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.725+0000 7f109875b640 1 --2- 172.21.15.92:0/749895278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1090100720 0x7f1090100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 -- 172.21.15.92:0/749895278 >> 172.21.15.92:0/749895278 conn(0x7f10900fbcb0 msgr2=0x7f10900fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 -- 172.21.15.92:0/749895278 shutdown_connections 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 -- 172.21.15.92:0/749895278 wait complete. 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 Processor -- start 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 -- start start 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1090100720 0x7f1090199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1090101910 0x7f109019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10901022d0 0x7f10901a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f109010c3a0 con 0x7f1090101910 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f109010c220 con 0x7f1090100720 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f109875b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f109010c520 con 0x7f10901022d0 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f1095cf6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1090101910 0x7f109019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f10964f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1090100720 0x7f1090199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f1095cf6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1090101910 0x7f109019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36986/0 (socket says 172.21.15.92:36986) 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f10964f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1090100720 0x7f1090199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:42740/0 (socket says 172.21.15.92:42740) 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f1095cf6640 1 -- 172.21.15.92:0/3510090740 learned_addr learned my addr 172.21.15.92:0/3510090740 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.726+0000 7f1096cf8640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10901022d0 0x7f10901a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f1095cf6640 1 -- 172.21.15.92:0/3510090740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10901022d0 msgr2=0x7f10901a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10901022d0 0x7f10901a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f1095cf6640 1 -- 172.21.15.92:0/3510090740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1090100720 msgr2=0x7f1090199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1090100720 0x7f1090199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f1095cf6640 1 -- 172.21.15.92:0/3510090740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10901a1960 con 0x7f1090101910 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f10964f7640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1090100720 0x7f1090199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1090101910 0x7f109019a280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f108000e9f0 tx=0x7f108000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f10877fe640 1 -- 172.21.15.92:0/3510090740 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108000cd80 con 0x7f1090101910 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f10877fe640 1 -- 172.21.15.92:0/3510090740 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1080005500 con 0x7f1090101910 2024-03-19T01:52:21.730 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f109875b640 1 -- 172.21.15.92:0/3510090740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10901a1c50 con 0x7f1090101910 2024-03-19T01:52:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f10877fe640 1 -- 172.21.15.92:0/3510090740 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080010690 con 0x7f1090101910 2024-03-19T01:52:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.727+0000 7f109875b640 1 -- 172.21.15.92:0/3510090740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10901a21a0 con 0x7f1090101910 2024-03-19T01:52:21.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.729+0000 7f10877fe640 1 -- 172.21.15.92:0/3510090740 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f10800046c0 con 0x7f1090101910 2024-03-19T01:52:21.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.730+0000 7f10877fe640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f107c0a2880 0x7f107c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.730+0000 7f10964f7640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f107c0a2880 0x7f107c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.730+0000 7f10877fe640 1 -- 172.21.15.92:0/3510090740 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f10800cedb0 con 0x7f1090101910 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.730+0000 7f10964f7640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f107c0a2880 0x7f107c0a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1090101770 tx=0x7f108c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.730+0000 7f109b422740 1 Processor -- start 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f109b422740 1 -- start start 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f109b422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2cfff0 0x5612af2d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f109b422740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f109b422740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d5b50 0x5612af2d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f109b422740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5612af234de0 con 0x5612af2d09e0 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f109b422740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5612af1ddc80 con 0x5612af2d5b50 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f109b422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5612af230e80 con 0x5612af2cfff0 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1095cf6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.738 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f10964f7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2cfff0 0x5612af2d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1095cf6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2d54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36990/0 (socket says 172.21.15.92:36990) 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1095cf6640 1 -- 172.21.15.92:0/353424703 learned_addr learned my addr 172.21.15.92:0/353424703 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1096cf8640 1 --2- 172.21.15.92:0/353424703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d5b50 0x5612af2d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1095cf6640 1 -- 172.21.15.92:0/353424703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2cfff0 msgr2=0x5612af2d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1095cf6640 1 --2- 172.21.15.92:0/353424703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2cfff0 0x5612af2d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1095cf6640 1 -- 172.21.15.92:0/353424703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d5b50 msgr2=0x5612af2d7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1095cf6640 1 --2- 172.21.15.92:0/353424703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d5b50 0x5612af2d7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1095cf6640 1 -- 172.21.15.92:0/353424703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612af1effb0 con 0x5612af2d09e0 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f10964f7640 1 --2- 172.21.15.92:0/353424703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2cfff0 0x5612af2d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1095cf6640 1 --2- 172.21.15.92:0/353424703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2d54e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1080097bc0 tx=0x7f108009b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4345ab2e18e96c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.731+0000 7f1084ff9640 1 -- 172.21.15.92:0/353424703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800b4070 con 0x5612af2d09e0 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f1084ff9640 1 -- 172.21.15.92:0/353424703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108009b760 con 0x5612af2d09e0 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f1084ff9640 1 -- 172.21.15.92:0/353424703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800a3a70 con 0x5612af2d09e0 2024-03-19T01:52:21.739 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f109b422740 1 -- 172.21.15.92:0/353424703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 msgr2=0x5612af2d54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f109b422740 1 --2- 172.21.15.92:0/353424703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2d54e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1080097bc0 tx=0x7f108009b2a0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f109b422740 1 -- 172.21.15.92:0/353424703 shutdown_connections 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f109b422740 1 --2- 172.21.15.92:0/353424703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d5b50 0x5612af2d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f109b422740 1 --2- 172.21.15.92:0/353424703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2d54e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f109b422740 1 --2- 172.21.15.92:0/353424703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2cfff0 0x5612af2d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f109b422740 1 -- 172.21.15.92:0/353424703 >> 172.21.15.92:0/353424703 conn(0x5612af2cb560 msgr2=0x5612af2cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f109b422740 1 -- 172.21.15.92:0/353424703 shutdown_connections 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.732+0000 7f109b422740 1 -- 172.21.15.92:0/353424703 wait complete. 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f109b422740 1 -- start start 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f109b422740 1 Processor -- start 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f109b422740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 0x5612af2da480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f109b422740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2da9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f109b422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 0x5612af2e1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.740 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f109b422740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5612af2d8a40 con 0x5612af2d09e0 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f109b422740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5612af1f0fc0 con 0x5612af2cfff0 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f109b422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5612af2d8bc0 con 0x5612af2d5b50 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f109b422740 10 client.0 _ll_register_callbacks cb 0x5612af22ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1095cf6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2da9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1095cf6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2da9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36994/0 (socket says 172.21.15.92:36994) 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f10964f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 0x5612af2da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1095cf6640 1 -- 172.21.15.92:0/3703447889 learned_addr learned my addr 172.21.15.92:0/3703447889 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1096cf8640 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 0x5612af2e1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1095cf6640 1 -- 172.21.15.92:0/3703447889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 msgr2=0x5612af2e1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 0x5612af2e1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1095cf6640 1 -- 172.21.15.92:0/3703447889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 msgr2=0x5612af2da480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 0x5612af2da480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1095cf6640 1 -- 172.21.15.92:0/3703447889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612aef492d0 con 0x5612af2d09e0 2024-03-19T01:52:21.741 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2da9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10800cc910 tx=0x7f10800860f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e5cf46357f6a82 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1079ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.733+0000 7f1079ffb640 1 -- 172.21.15.92:0/3703447889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800a9060 con 0x5612af2d09e0 2024-03-19T01:52:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f1079ffb640 1 -- 172.21.15.92:0/3703447889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f10800a3a70 con 0x5612af2d09e0 2024-03-19T01:52:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f109b422740 1 -- 172.21.15.92:0/3703447889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108009b760 con 0x5612af2d09e0 2024-03-19T01:52:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f1079ffb640 1 -- 172.21.15.92:0/3703447889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f10800b9070 con 0x5612af2d09e0 2024-03-19T01:52:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f109b422740 10 client.15021 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f109b422740 1 -- 172.21.15.92:0/3703447889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612af0136b0 con 0x5612af2d09e0 2024-03-19T01:52:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f1079ffb640 1 -- 172.21.15.92:0/3703447889 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108009b760 con 0x5612af2d09e0 2024-03-19T01:52:21.742 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f1079ffb640 1 -- 172.21.15.92:0/3703447889 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f108009ac40 con 0x5612af2d09e0 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f109b422740 10 client.15021 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f109b422740 10 client.15021 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f109b422740 20 client.15021 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f109b422740 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x5612af2e6ce0 0x5612af2e8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f109b422740 4 client.15021 mds_command: new command op to 14601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f109b422740 1 -- 172.21.15.92:0/3703447889 --> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612af242320 con 0x5612af2e6ce0 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f10964f7640 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x5612af2e6ce0 0x5612af2e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f1079ffb640 10 client.15021 ms_handle_connect on v2:172.21.15.92:6840/133114375 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.734+0000 7f10964f7640 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x5612af2e6ce0 0x5612af2e8150 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc0e9521528a0e46 server_cookie=a9dc73ffe918436d in_seq=0 out_seq=0 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f1079ffb640 1 -- 172.21.15.92:0/3703447889 <== mds.4 v2:172.21.15.92:6840/133114375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5612af242320 con 0x5612af2e6ce0 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f1079ffb640 10 client.15021 handle_command_reply: tid=0 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 client.15021 shutdown 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 -- 172.21.15.92:0/3703447889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 msgr2=0x5612af2da9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.743 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2da9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10800cc910 tx=0x7f10800860f0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 -- 172.21.15.92:0/3703447889 shutdown_connections 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x5612af2e6ce0 0x5612af2e8150 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 0x5612af2e1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d09e0 0x5612af2da9c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f1079ffb640 0 client.15021 ms_handle_reset on v2:172.21.15.92:6840/133114375 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 --2- 172.21.15.92:0/3703447889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 0x5612af2da480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 -- 172.21.15.92:0/3703447889 >> 172.21.15.92:0/3703447889 conn(0x5612af2cb560 msgr2=0x5612af2ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 -- 172.21.15.92:0/3703447889 shutdown_connections 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.735+0000 7f109b422740 1 -- 172.21.15.92:0/3703447889 wait complete. 2024-03-19T01:52:21.744 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.736+0000 7f109b422740 20 client.15021 trim_cache size 0 max 16384 2024-03-19T01:52:21.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.753+0000 7f109b422740 1 Processor -- start 2024-03-19T01:52:21.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.753+0000 7f109b422740 1 -- start start 2024-03-19T01:52:21.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.753+0000 7f109b422740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 0x5612af2e2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.753+0000 7f109b422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d09e0 0x5612af2e19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.759 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.753+0000 7f109b422740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d5b50 0x5612af2da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.753+0000 7f109b422740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5612af2e6490 con 0x5612af2d5b50 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.753+0000 7f109b422740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5612af2e6b60 con 0x5612af2cfff0 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.753+0000 7f109b422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5612af2e7230 con 0x5612af2d09e0 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1096cf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d5b50 0x5612af2da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1096cf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d5b50 0x5612af2da090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37004/0 (socket says 172.21.15.92:37004) 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1096cf8640 1 -- 172.21.15.92:0/238635677 learned_addr learned my addr 172.21.15.92:0/238635677 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f10964f7640 1 --2- 172.21.15.92:0/238635677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 0x5612af2e2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1095cf6640 1 --2- 172.21.15.92:0/238635677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d09e0 0x5612af2e19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1096cf8640 1 -- 172.21.15.92:0/238635677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d09e0 msgr2=0x5612af2e19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1096cf8640 1 --2- 172.21.15.92:0/238635677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d09e0 0x5612af2e19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1096cf8640 1 -- 172.21.15.92:0/238635677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 msgr2=0x5612af2e2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1096cf8640 1 --2- 172.21.15.92:0/238635677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 0x5612af2e2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1096cf8640 1 -- 172.21.15.92:0/238635677 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612af2d0dc0 con 0x5612af2d5b50 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f1096cf8640 1 --2- 172.21.15.92:0/238635677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d5b50 0x5612af2da090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f10880324c0 tx=0x7f1088032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c079a7a9e49846 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f107bfff640 1 -- 172.21.15.92:0/238635677 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088039040 con 0x5612af2d5b50 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f107bfff640 1 -- 172.21.15.92:0/238635677 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1088032c90 con 0x5612af2d5b50 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.754+0000 7f107bfff640 1 -- 172.21.15.92:0/238635677 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088004040 con 0x5612af2d5b50 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 -- 172.21.15.92:0/238635677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d5b50 msgr2=0x5612af2da090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 --2- 172.21.15.92:0/238635677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d5b50 0x5612af2da090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f10880324c0 tx=0x7f1088032590 comp rx=0 tx=0).stop 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 -- 172.21.15.92:0/238635677 shutdown_connections 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 --2- 172.21.15.92:0/238635677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2d5b50 0x5612af2da090 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 --2- 172.21.15.92:0/238635677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d09e0 0x5612af2e19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 --2- 172.21.15.92:0/238635677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2cfff0 0x5612af2e2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 -- 172.21.15.92:0/238635677 >> 172.21.15.92:0/238635677 conn(0x5612af2cb560 msgr2=0x5612af2d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 -- 172.21.15.92:0/238635677 shutdown_connections 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 -- 172.21.15.92:0/238635677 wait complete. 2024-03-19T01:52:21.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 -- start start 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 Processor -- start 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2cfff0 0x5612af2ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d09e0 0x5612af2de080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 0x5612af2e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5612af2e7fd0 con 0x5612af2cfff0 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5612af2e7900 con 0x5612af2d09e0 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5612af2e86a0 con 0x5612af2d5b50 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f109b422740 10 client.0 _ll_register_callbacks cb 0x7f10800b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f10964f7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2cfff0 0x5612af2ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f10964f7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2cfff0 0x5612af2ddb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37012/0 (socket says 172.21.15.92:37012) 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f10964f7640 1 -- 172.21.15.92:0/3975442256 learned_addr learned my addr 172.21.15.92:0/3975442256 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d09e0 0x5612af2de080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.755+0000 7f1096cf8640 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 0x5612af2e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f10964f7640 1 -- 172.21.15.92:0/3975442256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 msgr2=0x5612af2e10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f10964f7640 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 0x5612af2e10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f10964f7640 1 -- 172.21.15.92:0/3975442256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d09e0 msgr2=0x5612af2de080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f10964f7640 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d09e0 0x5612af2de080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f10964f7640 1 -- 172.21.15.92:0/3975442256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612af2e1690 con 0x5612af2cfff0 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f1096cf8640 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 0x5612af2e10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d09e0 0x5612af2de080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f10964f7640 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2cfff0 0x5612af2ddb40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f108c009460 tx=0x7f108c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05a84d6cc2833d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f1079ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f1079ffb640 1 -- 172.21.15.92:0/3975442256 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c003d70 con 0x5612af2cfff0 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f1079ffb640 1 -- 172.21.15.92:0/3975442256 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f108c005020 con 0x5612af2cfff0 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f109b422740 1 -- 172.21.15.92:0/3975442256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1088032c90 con 0x5612af2cfff0 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f1079ffb640 1 -- 172.21.15.92:0/3975442256 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f108c005220 con 0x5612af2cfff0 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f109b422740 10 client.15027 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f109b422740 1 -- 172.21.15.92:0/3975442256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612af34d520 con 0x5612af2cfff0 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f1079ffb640 1 -- 172.21.15.92:0/3975442256 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1088032c90 con 0x5612af2cfff0 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f1079ffb640 1 -- 172.21.15.92:0/3975442256 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f108c004830 con 0x5612af2cfff0 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f109b422740 10 client.15027 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f109b422740 10 client.15027 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f109b422740 20 client.15027 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f109b422740 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x5612af352130 0x5612af350e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f109b422740 4 client.15027 mds_command: new command op to 14601 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.756+0000 7f109b422740 1 -- 172.21.15.92:0/3975442256 --> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] -- command(tid 0: {"prefix": "status"}) -- 0x5612af23c670 con 0x5612af352130 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x5612af352130 0x5612af350e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f1079ffb640 10 client.15027 ms_handle_connect on v2:172.21.15.92:6840/133114375 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f1095cf6640 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x5612af352130 0x5612af350e40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=741871fa11676b3 server_cookie=c9061cefbc964a6b in_seq=0 out_seq=0 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f1079ffb640 1 -- 172.21.15.92:0/3975442256 <== mds.4 v2:172.21.15.92:6840/133114375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5612af23c670 con 0x5612af352130 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f1079ffb640 10 client.15027 handle_command_reply: tid=0 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 client.15027 shutdown 2024-03-19T01:52:21.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 -- 172.21.15.92:0/3975442256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2cfff0 msgr2=0x5612af2ddb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2cfff0 0x5612af2ddb40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f108c009460 tx=0x7f108c004730 comp rx=0 tx=0).stop 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 -- 172.21.15.92:0/3975442256 shutdown_connections 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x5612af352130 0x5612af350e40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5612af2d5b50 0x5612af2e10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612af2d09e0 0x5612af2de080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f1079ffb640 0 client.15027 ms_handle_reset on v2:172.21.15.92:6840/133114375 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 --2- 172.21.15.92:0/3975442256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612af2cfff0 0x5612af2ddb40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 -- 172.21.15.92:0/3975442256 >> 172.21.15.92:0/3975442256 conn(0x5612af2cb560 msgr2=0x5612af2da090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 -- 172.21.15.92:0/3975442256 shutdown_connections 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 1 -- 172.21.15.92:0/3975442256 wait complete. 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.757+0000 7f109b422740 20 client.15027 trim_cache size 0 max 16384 2024-03-19T01:52:21.765 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 4, 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 14601, 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 85.790445234000003, 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 100.52105802200001 2024-03-19T01:52:21.766 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 -- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f107c0a2880 msgr2=0x7f107c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f107c0a2880 0x7f107c0a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1090101770 tx=0x7f108c005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 -- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1090101910 msgr2=0x7f109019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.767 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1090101910 0x7f109019a280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f108000e9f0 tx=0x7f108000edf0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 -- 172.21.15.92:0/3510090740 shutdown_connections 2024-03-19T01:52:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f107c0a2880 0x7f107c0a4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10901022d0 0x7f10901a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1090101910 0x7f109019a280 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 --2- 172.21.15.92:0/3510090740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1090100720 0x7f1090199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 -- 172.21.15.92:0/3510090740 >> 172.21.15.92:0/3510090740 conn(0x7f10900fbcb0 msgr2=0x7f1090068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 -- 172.21.15.92:0/3510090740 shutdown_connections 2024-03-19T01:52:21.768 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.758+0000 7f109875b640 1 -- 172.21.15.92:0/3510090740 wait complete. 2024-03-19T01:52:21.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-19T01:52:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.881+0000 7f2548061640 1 -- 172.21.15.92:0/4092874193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25401033e0 msgr2=0x7f2540109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.881+0000 7f2548061640 1 --2- 172.21.15.92:0/4092874193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25401033e0 0x7f2540109c70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2538009ca0 tx=0x7f253802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.881+0000 7f2548061640 1 -- 172.21.15.92:0/4092874193 shutdown_connections 2024-03-19T01:52:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.881+0000 7f2548061640 1 --2- 172.21.15.92:0/4092874193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25401033e0 0x7f2540109c70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.881+0000 7f2548061640 1 --2- 172.21.15.92:0/4092874193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540102a20 0x7f2540102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.881+0000 7f2548061640 1 --2- 172.21.15.92:0/4092874193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2540101830 0x7f2540101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.881+0000 7f2548061640 1 -- 172.21.15.92:0/4092874193 >> 172.21.15.92:0/4092874193 conn(0x7f25400fcd80 msgr2=0x7f25400ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 -- 172.21.15.92:0/4092874193 shutdown_connections 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 -- 172.21.15.92:0/4092874193 wait complete. 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 Processor -- start 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 -- start start 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101830 0x7f2540197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2540102a20 0x7f2540198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25401033e0 0x7f254019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f254010d4b0 con 0x7f2540102a20 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f254010d330 con 0x7f2540101830 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2548061640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f254010d630 con 0x7f25401033e0 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f25455fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2540102a20 0x7f2540198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f25455fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2540102a20 0x7f2540198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37024/0 (socket says 172.21.15.92:37024) 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f25455fc640 1 -- 172.21.15.92:0/4233596032 learned_addr learned my addr 172.21.15.92:0/4233596032 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f25465fe640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25401033e0 0x7f254019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.882+0000 7f2545dfd640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101830 0x7f2540197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f25455fc640 1 -- 172.21.15.92:0/4233596032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25401033e0 msgr2=0x7f254019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f25455fc640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25401033e0 0x7f254019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f25455fc640 1 -- 172.21.15.92:0/4233596032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101830 msgr2=0x7f2540197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f25455fc640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101830 0x7f2540197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f25455fc640 1 -- 172.21.15.92:0/4233596032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254019f740 con 0x7f2540102a20 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f25455fc640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2540102a20 0x7f2540198060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f253400e9f0 tx=0x7f253400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f2532ffd640 1 -- 172.21.15.92:0/4233596032 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253400cd80 con 0x7f2540102a20 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f2548061640 1 -- 172.21.15.92:0/4233596032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254019fa30 con 0x7f2540102a20 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f2532ffd640 1 -- 172.21.15.92:0/4233596032 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2534010910 con 0x7f2540102a20 2024-03-19T01:52:21.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.883+0000 7f2532ffd640 1 -- 172.21.15.92:0/4233596032 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25340188f0 con 0x7f2540102a20 2024-03-19T01:52:21.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.884+0000 7f2548061640 1 -- 172.21.15.92:0/4233596032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254019ff80 con 0x7f2540102a20 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.885+0000 7f2532ffd640 1 -- 172.21.15.92:0/4233596032 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f2534010ac0 con 0x7f2540102a20 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f254ac51740 1 Processor -- start 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f254ac51740 1 -- start start 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f254ac51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f2532ffd640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f252c0a2800 0x7f252c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f254ac51740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467e99e0 0x5624467ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f254ac51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467eeb50 0x5624467f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f254ac51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56244674dde0 con 0x5624467e8ff0 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f254ac51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5624466f6c80 con 0x5624467eeb50 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f254ac51740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562446749e80 con 0x5624467e99e0 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f25465fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.893 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f2545dfd640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f252c0a2800 0x7f252c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f25455fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467e99e0 0x5624467ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f25465fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37038/0 (socket says 172.21.15.92:37038) 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.886+0000 7f25465fe640 1 -- 172.21.15.92:0/500123204 learned_addr learned my addr 172.21.15.92:0/500123204 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f2532ffd640 1 -- 172.21.15.92:0/4233596032 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f25340cee00 con 0x7f2540102a20 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f25465fe640 1 --2- 172.21.15.92:0/500123204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467eeb50 0x5624467f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f25465fe640 1 -- 172.21.15.92:0/500123204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467e99e0 msgr2=0x5624467ee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f25465fe640 1 --2- 172.21.15.92:0/500123204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467e99e0 0x5624467ee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f25465fe640 1 -- 172.21.15.92:0/500123204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467eeb50 msgr2=0x5624467f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f25465fe640 1 --2- 172.21.15.92:0/500123204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467eeb50 0x5624467f0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f25465fe640 1 -- 172.21.15.92:0/500123204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562446708fb0 con 0x5624467e8ff0 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f25455fc640 1 --2- 172.21.15.92:0/500123204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467e99e0 0x5624467ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f2545dfd640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f252c0a2800 0x7f252c0a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2540102880 tx=0x7f253c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f25465fe640 1 --2- 172.21.15.92:0/500123204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e9410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2538006010 tx=0x7f2538009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5adda227b105d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f252bfff640 1 -- 172.21.15.92:0/500123204 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538048070 con 0x5624467e8ff0 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f252bfff640 1 -- 172.21.15.92:0/500123204 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2538003970 con 0x5624467e8ff0 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.887+0000 7f252bfff640 1 -- 172.21.15.92:0/500123204 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538039040 con 0x5624467e8ff0 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 -- 172.21.15.92:0/500123204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 msgr2=0x5624467e9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 --2- 172.21.15.92:0/500123204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e9410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2538006010 tx=0x7f2538009360 comp rx=0 tx=0).stop 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 -- 172.21.15.92:0/500123204 shutdown_connections 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 --2- 172.21.15.92:0/500123204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467eeb50 0x5624467f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 --2- 172.21.15.92:0/500123204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467e99e0 0x5624467ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 --2- 172.21.15.92:0/500123204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e9410 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 -- 172.21.15.92:0/500123204 >> 172.21.15.92:0/500123204 conn(0x5624467e4560 msgr2=0x5624467e4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 -- 172.21.15.92:0/500123204 shutdown_connections 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 -- 172.21.15.92:0/500123204 wait complete. 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 -- start start 2024-03-19T01:52:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.888+0000 7f254ac51740 1 Processor -- start 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467faa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5624467f1aa0 con 0x5624467e8ff0 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562446709fc0 con 0x5624467e99e0 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5624467f1c20 con 0x5624467eeb50 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 10 client.0 _ll_register_callbacks cb 0x562446745a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f2545dfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f2545dfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37040/0 (socket says 172.21.15.92:37040) 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f2545dfd640 1 -- 172.21.15.92:0/4021728225 learned_addr learned my addr 172.21.15.92:0/4021728225 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f25465fe640 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f25455fc640 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f2545dfd640 1 -- 172.21.15.92:0/4021728225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 msgr2=0x5624467faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f2545dfd640 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467faa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f2545dfd640 1 -- 172.21.15.92:0/4021728225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 msgr2=0x5624467f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f2545dfd640 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467f3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f2545dfd640 1 -- 172.21.15.92:0/4021728225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624464622d0 con 0x5624467e8ff0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f2545dfd640 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f34e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f253c0097b0 tx=0x7f253c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878657f9ebdcb2d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f25297fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f25297fa640 1 -- 172.21.15.92:0/4021728225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c004220 con 0x5624467e8ff0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f25297fa640 1 -- 172.21.15.92:0/4021728225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f253c0043c0 con 0x5624467e8ff0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 1 -- 172.21.15.92:0/4021728225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2538003970 con 0x5624467e8ff0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f25297fa640 1 -- 172.21.15.92:0/4021728225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f253c01c7a0 con 0x5624467e8ff0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 10 client.15039 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.889+0000 7f254ac51740 1 -- 172.21.15.92:0/4021728225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56244652c6b0 con 0x5624467e8ff0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f25297fa640 1 -- 172.21.15.92:0/4021728225 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2538003970 con 0x5624467e8ff0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f25297fa640 1 -- 172.21.15.92:0/4021728225 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f253c027430 con 0x5624467e8ff0 2024-03-19T01:52:21.897 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f254ac51740 10 client.15039 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f254ac51740 10 client.15039 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f254ac51740 20 client.15039 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f254ac51740 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x562446800360 0x5624468017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f254ac51740 4 client.15039 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f254ac51740 1 -- 172.21.15.92:0/4021728225 --> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56244675b320 con 0x562446800360 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f25455fc640 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x562446800360 0x5624468017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f25297fa640 10 client.15039 ms_handle_connect on v2:172.21.15.159:6834/610100257 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.890+0000 7f25455fc640 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x562446800360 0x5624468017d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e163f0654aeae586 server_cookie=e4e77e13a0722912 in_seq=0 out_seq=0 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f25297fa640 1 -- 172.21.15.92:0/4021728225 <== mds.3 v2:172.21.15.159:6834/610100257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56244675b320 con 0x562446800360 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f25297fa640 10 client.15039 handle_command_reply: tid=0 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 client.15039 shutdown 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 -- 172.21.15.92:0/4021728225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 msgr2=0x5624467f34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f34e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f253c0097b0 tx=0x7f253c004730 comp rx=0 tx=0).stop 2024-03-19T01:52:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 -- 172.21.15.92:0/4021728225 shutdown_connections 2024-03-19T01:52:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x562446800360 0x5624468017d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 --2- 172.21.15.92:0/4021728225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f34e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f25297fa640 0 client.15039 ms_handle_reset on v2:172.21.15.159:6834/610100257 2024-03-19T01:52:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 -- 172.21.15.92:0/4021728225 >> 172.21.15.92:0/4021728225 conn(0x5624467e4560 msgr2=0x5624467e37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.891+0000 7f254ac51740 1 -- 172.21.15.92:0/4021728225 shutdown_connections 2024-03-19T01:52:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.892+0000 7f254ac51740 1 -- 172.21.15.92:0/4021728225 wait complete. 2024-03-19T01:52:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.892+0000 7f254ac51740 20 client.15039 trim_cache size 0 max 16384 2024-03-19T01:52:21.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.909+0000 7f254ac51740 1 Processor -- start 2024-03-19T01:52:21.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.909+0000 7f254ac51740 1 -- start start 2024-03-19T01:52:21.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f254ac51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f254ac51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f254ac51740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f254ac51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5624467ffb10 con 0x5624467e8ff0 2024-03-19T01:52:21.911 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f254ac51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5624468001e0 con 0x5624467e99e0 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f254ac51740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5624468008b0 con 0x5624467eeb50 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f2545dfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f2545dfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37046/0 (socket says 172.21.15.92:37046) 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f2545dfd640 1 -- 172.21.15.92:0/2072682761 learned_addr learned my addr 172.21.15.92:0/2072682761 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f25465fe640 1 --2- 172.21.15.92:0/2072682761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f25455fc640 1 --2- 172.21.15.92:0/2072682761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f2545dfd640 1 -- 172.21.15.92:0/2072682761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 msgr2=0x5624467eaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f2545dfd640 1 --2- 172.21.15.92:0/2072682761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467eaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f2545dfd640 1 -- 172.21.15.92:0/2072682761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 msgr2=0x5624467fa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f2545dfd640 1 --2- 172.21.15.92:0/2072682761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467fa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f2545dfd640 1 -- 172.21.15.92:0/2072682761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624467df670 con 0x5624467e8ff0 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f2545dfd640 1 --2- 172.21.15.92:0/2072682761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e4940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f253c004700 tx=0x7f253c011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b277b0ead63b3d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.912 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f252b7fe640 1 -- 172.21.15.92:0/2072682761 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c027d10 con 0x5624467e8ff0 2024-03-19T01:52:21.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f252b7fe640 1 -- 172.21.15.92:0/2072682761 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f253c01ce20 con 0x5624467e8ff0 2024-03-19T01:52:21.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.910+0000 7f252b7fe640 1 -- 172.21.15.92:0/2072682761 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c032ab0 con 0x5624467e8ff0 2024-03-19T01:52:21.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 -- 172.21.15.92:0/2072682761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 msgr2=0x5624467e4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 --2- 172.21.15.92:0/2072682761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e4940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f253c004700 tx=0x7f253c011950 comp rx=0 tx=0).stop 2024-03-19T01:52:21.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 -- 172.21.15.92:0/2072682761 shutdown_connections 2024-03-19T01:52:21.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 --2- 172.21.15.92:0/2072682761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467eaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 --2- 172.21.15.92:0/2072682761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467fa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.913 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 --2- 172.21.15.92:0/2072682761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467e4940 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 -- 172.21.15.92:0/2072682761 >> 172.21.15.92:0/2072682761 conn(0x5624467e4560 msgr2=0x5624467ea450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 -- 172.21.15.92:0/2072682761 shutdown_connections 2024-03-19T01:52:21.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 -- 172.21.15.92:0/2072682761 wait complete. 2024-03-19T01:52:21.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 -- start start 2024-03-19T01:52:21.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.911+0000 7f254ac51740 1 Processor -- start 2024-03-19T01:52:21.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.917 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562446801650 con 0x5624467e8ff0 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562446800f80 con 0x5624467e99e0 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562446801d20 con 0x5624467eeb50 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 10 client.0 _ll_register_callbacks cb 0x7f253c005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f2545dfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f2545dfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37056/0 (socket says 172.21.15.92:37056) 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f2545dfd640 1 -- 172.21.15.92:0/261644056 learned_addr learned my addr 172.21.15.92:0/261644056 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f25465fe640 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f25455fc640 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f2545dfd640 1 -- 172.21.15.92:0/261644056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 msgr2=0x5624467f9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f2545dfd640 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467f9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f2545dfd640 1 -- 172.21.15.92:0/261644056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 msgr2=0x5624467f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:21.918 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f2545dfd640 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f2545dfd640 1 -- 172.21.15.92:0/261644056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624467fa070 con 0x5624467e8ff0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f2545dfd640 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f6bb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f253c016040 tx=0x7f253c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb80ff2d15e18db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f25297fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f25297fa640 1 -- 172.21.15.92:0/261644056 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c049060 con 0x5624467e8ff0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f25297fa640 1 -- 172.21.15.92:0/261644056 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f253c027ca0 con 0x5624467e8ff0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 1 -- 172.21.15.92:0/261644056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f253c01ce20 con 0x5624467e8ff0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f25297fa640 1 -- 172.21.15.92:0/261644056 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f253c01c940 con 0x5624467e8ff0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 10 client.15045 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.912+0000 7f254ac51740 1 -- 172.21.15.92:0/261644056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624468663f0 con 0x5624467e8ff0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f25297fa640 1 -- 172.21.15.92:0/261644056 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f253c01ce20 con 0x5624467e8ff0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f25297fa640 1 -- 172.21.15.92:0/261644056 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f253c043070 con 0x5624467e8ff0 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f254ac51740 10 client.15045 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f254ac51740 10 client.15045 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-19T01:52:21.919 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f254ac51740 20 client.15045 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f254ac51740 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x56244686ac00 0x562446869910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f254ac51740 4 client.15045 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f254ac51740 1 -- 172.21.15.92:0/261644056 --> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] -- command(tid 0: {"prefix": "status"}) -- 0x562446755670 con 0x56244686ac00 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f25455fc640 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x56244686ac00 0x562446869910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f25297fa640 10 client.15045 ms_handle_connect on v2:172.21.15.159:6834/610100257 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.913+0000 7f25455fc640 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x56244686ac00 0x562446869910 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b7e55d838740ef1 server_cookie=1c7c5d63dfbf556 in_seq=0 out_seq=0 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f25297fa640 1 -- 172.21.15.92:0/261644056 <== mds.3 v2:172.21.15.159:6834/610100257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562446755670 con 0x56244686ac00 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f25297fa640 10 client.15045 handle_command_reply: tid=0 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 client.15045 shutdown 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 -- 172.21.15.92:0/261644056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 msgr2=0x5624467f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f6bb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f253c016040 tx=0x7f253c014000 comp rx=0 tx=0).stop 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 -- 172.21.15.92:0/261644056 shutdown_connections 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x56244686ac00 0x562446869910 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624467eeb50 0x5624467f9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5624467e99e0 0x5624467f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f25297fa640 0 client.15045 ms_handle_reset on v2:172.21.15.159:6834/610100257 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 --2- 172.21.15.92:0/261644056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5624467e8ff0 0x5624467f6bb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 -- 172.21.15.92:0/261644056 >> 172.21.15.92:0/261644056 conn(0x5624467e4560 msgr2=0x5624467e4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 -- 172.21.15.92:0/261644056 shutdown_connections 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 1 -- 172.21.15.92:0/261644056 wait complete. 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f254ac51740 20 client.15045 trim_cache size 0 max 16384 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:21.921 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 3, 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24457, 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 86.919642249000006, 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 115.05572456900001 2024-03-19T01:52:21.922 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 -- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f252c0a2800 msgr2=0x7f252c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f252c0a2800 0x7f252c0a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2540102880 tx=0x7f253c005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 -- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2540102a20 msgr2=0x7f2540198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2540102a20 0x7f2540198060 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f253400e9f0 tx=0x7f253400eeb0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 -- 172.21.15.92:0/4233596032 shutdown_connections 2024-03-19T01:52:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f252c0a2800 0x7f252c0a4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25401033e0 0x7f254019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2540102a20 0x7f2540198060 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 --2- 172.21.15.92:0/4233596032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2540101830 0x7f2540197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:21.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.914+0000 7f2548061640 1 -- 172.21.15.92:0/4233596032 >> 172.21.15.92:0/4233596032 conn(0x7f25400fcd80 msgr2=0x7f2540105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:21.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.915+0000 7f2548061640 1 -- 172.21.15.92:0/4233596032 shutdown_connections 2024-03-19T01:52:21.924 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:21.915+0000 7f2548061640 1 -- 172.21.15.92:0/4233596032 wait complete. 2024-03-19T01:52:21.924 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:21 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1220816400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:21.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:21 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2578226651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:21.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:21 smithi092 ceph-mon[62574]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:52:21.925 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:21 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3277445244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:21.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-19T01:52:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 -- 172.21.15.92:0/1005557657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f499c100110 msgr2=0x7f499c100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 --2- 172.21.15.92:0/1005557657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f499c100110 0x7f499c100530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4994009ca0 tx=0x7f499402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 -- 172.21.15.92:0/1005557657 shutdown_connections 2024-03-19T01:52:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 --2- 172.21.15.92:0/1005557657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f499c101410 0x7f499c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 --2- 172.21.15.92:0/1005557657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c100a70 0x7f499c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 --2- 172.21.15.92:0/1005557657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f499c100110 0x7f499c100530 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 -- 172.21.15.92:0/1005557657 >> 172.21.15.92:0/1005557657 conn(0x7f499c0fbcd0 msgr2=0x7f499c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 -- 172.21.15.92:0/1005557657 shutdown_connections 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 -- 172.21.15.92:0/1005557657 wait complete. 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 Processor -- start 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 -- start start 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f499c100110 0x7f499c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f499c100a70 0x7f499c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c101410 0x7f499c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f499c10f6a0 con 0x7f499c100110 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f499c10f520 con 0x7f499c101410 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.031+0000 7f49a2adc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f499c10f820 con 0x7f499c100a70 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a0878640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f499c100110 0x7f499c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a0878640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f499c100110 0x7f499c199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37066/0 (socket says 172.21.15.92:37066) 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a0878640 1 -- 172.21.15.92:0/112892237 learned_addr learned my addr 172.21.15.92:0/112892237 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f499bfff640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f499c100a70 0x7f499c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a1079640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c101410 0x7f499c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a0878640 1 -- 172.21.15.92:0/112892237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f499c100a70 msgr2=0x7f499c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a0878640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f499c100a70 0x7f499c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a0878640 1 -- 172.21.15.92:0/112892237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c101410 msgr2=0x7f499c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a0878640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c101410 0x7f499c1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a0878640 1 -- 172.21.15.92:0/112892237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c1a1990 con 0x7f499c100110 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a0878640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f499c100110 0x7f499c199d70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4994009dc0 tx=0x7f4994032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f4999ffb640 1 -- 172.21.15.92:0/112892237 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994041020 con 0x7f499c100110 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f49a2adc640 1 -- 172.21.15.92:0/112892237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c1a1c20 con 0x7f499c100110 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f4999ffb640 1 -- 172.21.15.92:0/112892237 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4994034cd0 con 0x7f499c100110 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.032+0000 7f4999ffb640 1 -- 172.21.15.92:0/112892237 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994039040 con 0x7f499c100110 2024-03-19T01:52:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.033+0000 7f49a2adc640 1 -- 172.21.15.92:0/112892237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c1a2160 con 0x7f499c100110 2024-03-19T01:52:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.035+0000 7f49a57a3740 1 Processor -- start 2024-03-19T01:52:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.035+0000 7f4999ffb640 1 -- 172.21.15.92:0/112892237 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4994044020 con 0x7f499c100110 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.035+0000 7f49a57a3740 1 -- start start 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.035+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f32b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f3304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 0x55a03f332fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a57a3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a03f28fde0 con 0x55a03f32b9e0 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a57a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a03f238c80 con 0x55a03f330b50 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a57a3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a03f28be80 con 0x55a03f32aff0 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a0878640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f3304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a1079640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f32b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a1079640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f32b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:47756/0 (socket says 172.21.15.92:47756) 2024-03-19T01:52:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a0878640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f3304e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37072/0 (socket says 172.21.15.92:37072) 2024-03-19T01:52:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a1079640 1 -- 172.21.15.92:0/4135237452 learned_addr learned my addr 172.21.15.92:0/4135237452 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f499bfff640 1 --2- 172.21.15.92:0/4135237452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 0x55a03f332fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a1079640 1 -- 172.21.15.92:0/4135237452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 msgr2=0x55a03f332fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a1079640 1 --2- 172.21.15.92:0/4135237452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 0x55a03f332fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a1079640 1 -- 172.21.15.92:0/4135237452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 msgr2=0x55a03f3304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a1079640 1 --2- 172.21.15.92:0/4135237452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f3304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a1079640 1 -- 172.21.15.92:0/4135237452 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03f24afb0 con 0x55a03f32aff0 2024-03-19T01:52:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f4999ffb640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f49880a2800 0x7f49880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a0878640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f49880a2800 0x7f49880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f49a1079640 1 --2- 172.21.15.92:0/4135237452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f32b410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f498c009c90 tx=0x7f498c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9717faaaf2633d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f4999ffb640 1 -- 172.21.15.92:0/112892237 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f49940f0860 con 0x7f499c100110 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f4992ffd640 1 -- 172.21.15.92:0/4135237452 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c004230 con 0x55a03f32aff0 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f4992ffd640 1 -- 172.21.15.92:0/4135237452 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c009e40 con 0x55a03f32aff0 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.036+0000 7f4992ffd640 1 -- 172.21.15.92:0/4135237452 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c019760 con 0x55a03f32aff0 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 -- 172.21.15.92:0/4135237452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 msgr2=0x55a03f32b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a0878640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f49880a2800 0x7f49880a4cc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f49940ee6e0 tx=0x7f49940f6550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4135237452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f32b410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f498c009c90 tx=0x7f498c011810 comp rx=0 tx=0).stop 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 -- 172.21.15.92:0/4135237452 shutdown_connections 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4135237452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 0x55a03f332fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4135237452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f3304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4135237452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f32b410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 -- 172.21.15.92:0/4135237452 >> 172.21.15.92:0/4135237452 conn(0x55a03f326560 msgr2=0x55a03f326960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 -- 172.21.15.92:0/4135237452 shutdown_connections 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 -- 172.21.15.92:0/4135237452 wait complete. 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 -- start start 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.037+0000 7f49a57a3740 1 Processor -- start 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32aff0 0x55a03f3354e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f335a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f330b50 0x55a03f33ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a57a3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a03f333b00 con 0x55a03f32b9e0 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a57a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a03f24bfc0 con 0x55a03f32aff0 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a57a3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a03f333c80 con 0x55a03f330b50 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a57a3740 10 client.0 _ll_register_callbacks cb 0x55a03f287a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f499bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f335a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a1079640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32aff0 0x55a03f3354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f499bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f335a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37076/0 (socket says 172.21.15.92:37076) 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f499bfff640 1 -- 172.21.15.92:0/4256722757 learned_addr learned my addr 172.21.15.92:0/4256722757 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a1079640 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f330b50 0x55a03f33ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f499bfff640 1 -- 172.21.15.92:0/4256722757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f330b50 msgr2=0x55a03f33ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f499bfff640 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f330b50 0x55a03f33ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f499bfff640 1 -- 172.21.15.92:0/4256722757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32aff0 msgr2=0x55a03f3354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f499bfff640 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32aff0 0x55a03f3354e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f499bfff640 1 -- 172.21.15.92:0/4256722757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03efa42d0 con 0x55a03f32b9e0 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f49a1079640 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32aff0 0x55a03f3354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f499bfff640 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f335a20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f49840126e0 tx=0x7f4984012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c45d5dd68a2965e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f497bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.038+0000 7f497bfff640 1 -- 172.21.15.92:0/4256722757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498400f8c0 con 0x55a03f32b9e0 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f497bfff640 1 -- 172.21.15.92:0/4256722757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f498400fa60 con 0x55a03f32b9e0 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a57a3740 1 -- 172.21.15.92:0/4256722757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a03f29d320 con 0x55a03f32b9e0 2024-03-19T01:52:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f497bfff640 1 -- 172.21.15.92:0/4256722757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f498400fc60 con 0x55a03f32b9e0 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a57a3740 10 client.15057 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a57a3740 1 -- 172.21.15.92:0/4256722757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a03f06e6b0 con 0x55a03f32b9e0 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f497bfff640 1 -- 172.21.15.92:0/4256722757 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a03f29d320 con 0x55a03f32b9e0 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f497bfff640 1 -- 172.21.15.92:0/4256722757 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f498401c420 con 0x55a03f32b9e0 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a57a3740 10 client.15057 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a57a3740 10 client.15057 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a57a3740 20 client.15057 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x55a03f3423b0 0x55a03f343820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a57a3740 4 client.15057 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a57a3740 1 -- 172.21.15.92:0/4256722757 --> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a03f297670 con 0x55a03f3423b0 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.039+0000 7f49a1079640 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x55a03f3423b0 0x55a03f343820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.040+0000 7f497bfff640 10 client.15057 ms_handle_connect on v2:172.21.15.159:6836/3903346711 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.040+0000 7f49a1079640 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x55a03f3423b0 0x55a03f343820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be5be7588df63b2e server_cookie=8b77d05820419c0a in_seq=0 out_seq=0 2024-03-19T01:52:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.040+0000 7f497bfff640 1 -- 172.21.15.92:0/4256722757 <== mds.2 v2:172.21.15.159:6836/3903346711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a03f297670 con 0x55a03f3423b0 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.040+0000 7f497bfff640 10 client.15057 handle_command_reply: tid=0 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.040+0000 7f49a57a3740 1 client.15057 shutdown 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 -- 172.21.15.92:0/4256722757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 msgr2=0x55a03f335a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f335a20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f49840126e0 tx=0x7f4984012ba0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 -- 172.21.15.92:0/4256722757 shutdown_connections 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x55a03f3423b0 0x55a03f343820 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f330b50 0x55a03f33ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f335a20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 --2- 172.21.15.92:0/4256722757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32aff0 0x55a03f3354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f497bfff640 0 client.15057 ms_handle_reset on v2:172.21.15.159:6836/3903346711 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 -- 172.21.15.92:0/4256722757 >> 172.21.15.92:0/4256722757 conn(0x55a03f326560 msgr2=0x55a03f3257e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 -- 172.21.15.92:0/4256722757 shutdown_connections 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 1 -- 172.21.15.92:0/4256722757 wait complete. 2024-03-19T01:52:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.041+0000 7f49a57a3740 20 client.15057 trim_cache size 0 max 16384 2024-03-19T01:52:22.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a57a3740 1 Processor -- start 2024-03-19T01:52:22.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a57a3740 1 -- start start 2024-03-19T01:52:22.065 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f3339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f33ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 0x55a03f32cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a57a3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a03f341b60 con 0x55a03f32b9e0 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a57a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a03f342230 con 0x55a03f330b50 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a57a3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a03f342900 con 0x55a03f32aff0 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f499bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f33ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f499bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f33ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37088/0 (socket says 172.21.15.92:37088) 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f499bfff640 1 -- 172.21.15.92:0/2642803468 learned_addr learned my addr 172.21.15.92:0/2642803468 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a1079640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f3339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49a1079640 1 --2- 172.21.15.92:0/2642803468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 0x55a03f32cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f499bfff640 1 -- 172.21.15.92:0/2642803468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 msgr2=0x55a03f3339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f499bfff640 1 --2- 172.21.15.92:0/2642803468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f3339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f499bfff640 1 -- 172.21.15.92:0/2642803468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 msgr2=0x55a03f32cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f499bfff640 1 --2- 172.21.15.92:0/2642803468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 0x55a03f32cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f499bfff640 1 -- 172.21.15.92:0/2642803468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03f32bdc0 con 0x55a03f32b9e0 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f499bfff640 1 --2- 172.21.15.92:0/2642803468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f33ca20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f498401ca80 tx=0x7f498400ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502e56efe3e34a38 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.059+0000 7f49927fc640 1 -- 172.21.15.92:0/2642803468 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498401cb00 con 0x55a03f32b9e0 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49927fc640 1 -- 172.21.15.92:0/2642803468 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4984022030 con 0x55a03f32b9e0 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49927fc640 1 -- 172.21.15.92:0/2642803468 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984010910 con 0x55a03f32b9e0 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 -- 172.21.15.92:0/2642803468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 msgr2=0x55a03f33ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 --2- 172.21.15.92:0/2642803468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f33ca20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f498401ca80 tx=0x7f498400ff90 comp rx=0 tx=0).stop 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 -- 172.21.15.92:0/2642803468 shutdown_connections 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 --2- 172.21.15.92:0/2642803468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f330b50 0x55a03f32cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 --2- 172.21.15.92:0/2642803468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f32b9e0 0x55a03f33ca20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 --2- 172.21.15.92:0/2642803468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f3339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 -- 172.21.15.92:0/2642803468 >> 172.21.15.92:0/2642803468 conn(0x55a03f326560 msgr2=0x55a03f326940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 -- 172.21.15.92:0/2642803468 shutdown_connections 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 -- 172.21.15.92:0/2642803468 wait complete. 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 -- start start 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.060+0000 7f49a57a3740 1 Processor -- start 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f33abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32b9e0 0x55a03f33b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a57a3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f330b50 0x55a03f33b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a57a3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a03f3436a0 con 0x55a03f330b50 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a57a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a03f342fd0 con 0x55a03f32b9e0 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a57a3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a03f343d70 con 0x55a03f32aff0 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a57a3740 10 client.0 _ll_register_callbacks cb 0x7f4984009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f330b50 0x55a03f33b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f330b50 0x55a03f33b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37094/0 (socket says 172.21.15.92:37094) 2024-03-19T01:52:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 -- 172.21.15.92:0/1437687087 learned_addr learned my addr 172.21.15.92:0/1437687087 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f499bfff640 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32b9e0 0x55a03f33b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f33abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 -- 172.21.15.92:0/1437687087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 msgr2=0x55a03f33abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f33abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 -- 172.21.15.92:0/1437687087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32b9e0 msgr2=0x55a03f33b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32b9e0 0x55a03f33b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 -- 172.21.15.92:0/1437687087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03f33be70 con 0x55a03f330b50 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f499bfff640 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32b9e0 0x55a03f33b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f49a1079640 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f330b50 0x55a03f33b640 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f498c000c00 tx=0x7f498c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cd1d50974d747e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.061+0000 7f497bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f497bfff640 1 -- 172.21.15.92:0/1437687087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c003930 con 0x55a03f330b50 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f497bfff640 1 -- 172.21.15.92:0/1437687087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f498c01f040 con 0x55a03f330b50 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f49a57a3740 1 -- 172.21.15.92:0/1437687087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4984022030 con 0x55a03f330b50 2024-03-19T01:52:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f497bfff640 1 -- 172.21.15.92:0/1437687087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f498c003ad0 con 0x55a03f330b50 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f49a57a3740 10 client.15063 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f49a57a3740 1 -- 172.21.15.92:0/1437687087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a03f3a82e0 con 0x55a03f330b50 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f497bfff640 1 -- 172.21.15.92:0/1437687087 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4984022030 con 0x55a03f330b50 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f497bfff640 1 -- 172.21.15.92:0/1437687087 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f498c025040 con 0x55a03f330b50 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f49a57a3740 10 client.15063 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f49a57a3740 10 client.15063 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f49a57a3740 20 client.15063 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f49a57a3740 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x55a03f3acd40 0x55a03f3aba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f49a57a3740 4 client.15063 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f49a57a3740 1 -- 172.21.15.92:0/1437687087 --> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] -- command(tid 0: {"prefix": "status"}) -- 0x55a03f215a00 con 0x55a03f3acd40 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.062+0000 7f499bfff640 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x55a03f3acd40 0x55a03f3aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f497bfff640 10 client.15063 ms_handle_connect on v2:172.21.15.159:6836/3903346711 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f499bfff640 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x55a03f3acd40 0x55a03f3aba50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed786f4645c4c1f4 server_cookie=e357956f08ccba6b in_seq=0 out_seq=0 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f497bfff640 1 -- 172.21.15.92:0/1437687087 <== mds.2 v2:172.21.15.159:6836/3903346711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a03f215a00 con 0x55a03f3acd40 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f497bfff640 10 client.15063 handle_command_reply: tid=0 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 client.15063 shutdown 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 -- 172.21.15.92:0/1437687087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f330b50 msgr2=0x55a03f33b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f330b50 0x55a03f33b640 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f498c000c00 tx=0x7f498c014040 comp rx=0 tx=0).stop 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 -- 172.21.15.92:0/1437687087 shutdown_connections 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a03f330b50 0x55a03f33b640 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a03f32b9e0 0x55a03f33b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x55a03f3acd40 0x55a03f3aba50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 --2- 172.21.15.92:0/1437687087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a03f32aff0 0x55a03f33abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 -- 172.21.15.92:0/1437687087 >> 172.21.15.92:0/1437687087 conn(0x55a03f326560 msgr2=0x55a03f32cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f497bfff640 0 client.15063 ms_handle_reset on v2:172.21.15.159:6836/3903346711 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 -- 172.21.15.92:0/1437687087 shutdown_connections 2024-03-19T01:52:22.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 1 -- 172.21.15.92:0/1437687087 wait complete. 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.063+0000 7f49a57a3740 20 client.15063 trim_cache size 0 max 16384 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24475, 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 88.075491811999996, 2024-03-19T01:52:22.072 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:22.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:22.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:22.073 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 112.299198901 2024-03-19T01:52:22.073 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 -- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f49880a2800 msgr2=0x7f49880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f49880a2800 0x7f49880a4cc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f49940ee6e0 tx=0x7f49940f6550 comp rx=0 tx=0).stop 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 -- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f499c100110 msgr2=0x7f499c199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f499c100110 0x7f499c199d70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4994009dc0 tx=0x7f4994032f20 comp rx=0 tx=0).stop 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 -- 172.21.15.92:0/112892237 shutdown_connections 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f499c101410 0x7f499c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f49880a2800 0x7f49880a4cc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f499c100a70 0x7f499c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 --2- 172.21.15.92:0/112892237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f499c100110 0x7f499c199d70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 -- 172.21.15.92:0/112892237 >> 172.21.15.92:0/112892237 conn(0x7f499c0fbcd0 msgr2=0x7f499c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 -- 172.21.15.92:0/112892237 shutdown_connections 2024-03-19T01:52:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.064+0000 7f49a2adc640 1 -- 172.21.15.92:0/112892237 wait complete. 2024-03-19T01:52:22.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-19T01:52:22.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:21 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1220816400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:22.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:21 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2578226651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:22.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:21 smithi144 ceph-mon[67199]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:52:22.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:21 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3277445244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.178+0000 7fb2345b3640 1 -- 172.21.15.92:0/940743728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c102340 msgr2=0x7fb22c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.178+0000 7fb2345b3640 1 --2- 172.21.15.92:0/940743728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c102340 0x7fb22c104730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb228009900 tx=0x7fb22802bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 -- 172.21.15.92:0/940743728 shutdown_connections 2024-03-19T01:52:22.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 --2- 172.21.15.92:0/940743728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb22c1075a0 0x7fb22c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 --2- 172.21.15.92:0/940743728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22c104c70 0x7fb22c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 --2- 172.21.15.92:0/940743728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c102340 0x7fb22c104730 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 -- 172.21.15.92:0/940743728 >> 172.21.15.92:0/940743728 conn(0x7fb22c0fbcb0 msgr2=0x7fb22c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 -- 172.21.15.92:0/940743728 shutdown_connections 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 -- 172.21.15.92:0/940743728 wait complete. 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 Processor -- start 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 -- start start 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22c102340 0x7fb22c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c104c70 0x7fb22c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb22c1075a0 0x7fb22c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c10d490 con 0x7fb22c104c70 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c10d310 con 0x7fb22c102340 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb2345b3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c10d610 con 0x7fb22c1075a0 2024-03-19T01:52:22.182 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb231b4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c104c70 0x7fb22c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb231b4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c104c70 0x7fb22c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37118/0 (socket says 172.21.15.92:37118) 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb231b4e640 1 -- 172.21.15.92:0/401978997 learned_addr learned my addr 172.21.15.92:0/401978997 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.179+0000 7fb232b50640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb22c1075a0 0x7fb22c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb23234f640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22c102340 0x7fb22c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb231b4e640 1 -- 172.21.15.92:0/401978997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb22c1075a0 msgr2=0x7fb22c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb231b4e640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb22c1075a0 0x7fb22c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb231b4e640 1 -- 172.21.15.92:0/401978997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22c102340 msgr2=0x7fb22c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb231b4e640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22c102340 0x7fb22c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb231b4e640 1 -- 172.21.15.92:0/401978997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c19f7b0 con 0x7fb22c104c70 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb231b4e640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c104c70 0x7fb22c1980d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e9a0 tx=0x7fb21c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb2237fe640 1 -- 172.21.15.92:0/401978997 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00cd80 con 0x7fb22c104c70 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb2345b3640 1 -- 172.21.15.92:0/401978997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22c19faa0 con 0x7fb22c104c70 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb2237fe640 1 -- 172.21.15.92:0/401978997 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb21c0035f0 con 0x7fb22c104c70 2024-03-19T01:52:22.183 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb2237fe640 1 -- 172.21.15.92:0/401978997 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c010670 con 0x7fb22c104c70 2024-03-19T01:52:22.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.180+0000 7fb2345b3640 1 -- 172.21.15.92:0/401978997 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22c19fff0 con 0x7fb22c104c70 2024-03-19T01:52:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.183+0000 7fb2237fe640 1 -- 172.21.15.92:0/401978997 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb21c0037a0 con 0x7fb22c104c70 2024-03-19T01:52:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.183+0000 7fb237199740 1 Processor -- start 2024-03-19T01:52:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.183+0000 7fb237199740 1 -- start start 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.183+0000 7fb237199740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 0x55f332e06410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.183+0000 7fb237199740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e069e0 0x55f332e0b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.183+0000 7fb237199740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e0bb50 0x55f332e0dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.183+0000 7fb237199740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f332d6ade0 con 0x55f332e0bb50 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.183+0000 7fb237199740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f332d13c80 con 0x55f332e05ff0 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb237199740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f332d66e80 con 0x55f332e069e0 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb231b4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e0bb50 0x55f332e0dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb2237fe640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2180a2800 0x7fb2180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb23234f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e069e0 0x55f332e0b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb23234f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e069e0 0x55f332e0b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:47810/0 (socket says 172.21.15.92:47810) 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb231b4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e0bb50 0x55f332e0dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37128/0 (socket says 172.21.15.92:37128) 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb23234f640 1 -- 172.21.15.92:0/3137415415 learned_addr learned my addr 172.21.15.92:0/3137415415 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb232b50640 1 --2- 172.21.15.92:0/3137415415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 0x55f332e06410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb232b50640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2180a2800 0x7fb2180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb23234f640 1 -- 172.21.15.92:0/3137415415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 msgr2=0x55f332e06410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb23234f640 1 --2- 172.21.15.92:0/3137415415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 0x55f332e06410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb2237fe640 1 -- 172.21.15.92:0/401978997 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb21c0cecb0 con 0x7fb22c104c70 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb23234f640 1 -- 172.21.15.92:0/3137415415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e0bb50 msgr2=0x55f332e0dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb23234f640 1 --2- 172.21.15.92:0/3137415415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e0bb50 0x55f332e0dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb23234f640 1 -- 172.21.15.92:0/3137415415 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f332d25fb0 con 0x55f332e069e0 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb232b50640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2180a2800 0x7fb2180a4cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55f332e07240 tx=0x7fb224009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb23234f640 1 --2- 172.21.15.92:0/3137415415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e069e0 0x55f332e0b4e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb228009900 tx=0x7fb228009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c697699fe2d482cc server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.184+0000 7fb220bfe640 1 -- 172.21.15.92:0/3137415415 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22802cb80 con 0x55f332e069e0 2024-03-19T01:52:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb220bfe640 1 -- 172.21.15.92:0/3137415415 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb228005040 con 0x55f332e069e0 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb220bfe640 1 -- 172.21.15.92:0/3137415415 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2280048e0 con 0x55f332e069e0 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb237199740 1 -- 172.21.15.92:0/3137415415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e069e0 msgr2=0x55f332e0b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb237199740 1 --2- 172.21.15.92:0/3137415415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e069e0 0x55f332e0b4e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb228009900 tx=0x7fb228009560 comp rx=0 tx=0).stop 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb237199740 1 -- 172.21.15.92:0/3137415415 shutdown_connections 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb237199740 1 --2- 172.21.15.92:0/3137415415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e0bb50 0x55f332e0dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb237199740 1 --2- 172.21.15.92:0/3137415415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e069e0 0x55f332e0b4e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb237199740 1 --2- 172.21.15.92:0/3137415415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 0x55f332e06410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb237199740 1 -- 172.21.15.92:0/3137415415 >> 172.21.15.92:0/3137415415 conn(0x55f332e01560 msgr2=0x55f332e01960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb237199740 1 -- 172.21.15.92:0/3137415415 shutdown_connections 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.185+0000 7fb237199740 1 -- 172.21.15.92:0/3137415415 wait complete. 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb237199740 1 -- start start 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb237199740 1 Processor -- start 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb237199740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 0x55f332e10590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb237199740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb237199740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 0x55f332e17aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb237199740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f332e0eaa0 con 0x55f332e069e0 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb237199740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f332d26fc0 con 0x55f332e05ff0 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb237199740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f332e0ec20 con 0x55f332e0bb50 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb237199740 10 client.0 _ll_register_callbacks cb 0x55f332d62a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb231b4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb231b4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37142/0 (socket says 172.21.15.92:37142) 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb231b4e640 1 -- 172.21.15.92:0/2146470559 learned_addr learned my addr 172.21.15.92:0/2146470559 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb232b50640 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 0x55f332e17aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb23234f640 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 0x55f332e10590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb231b4e640 1 -- 172.21.15.92:0/2146470559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 msgr2=0x55f332e17aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb231b4e640 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 0x55f332e17aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb231b4e640 1 -- 172.21.15.92:0/2146470559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 msgr2=0x55f332e10590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb231b4e640 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 0x55f332e10590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb231b4e640 1 -- 172.21.15.92:0/2146470559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f332a7f2d0 con 0x55f332e069e0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb23234f640 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 0x55f332e10590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb231b4e640 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e970 tx=0x7fb21c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc99d3527b9dc01e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.186+0000 7fb215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb215ffb640 1 -- 172.21.15.92:0/2146470559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c09bd60 con 0x55f332e069e0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb215ffb640 1 -- 172.21.15.92:0/2146470559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb21c097d30 con 0x55f332e069e0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb237199740 1 -- 172.21.15.92:0/2146470559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb228005040 con 0x55f332e069e0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb215ffb640 1 -- 172.21.15.92:0/2146470559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb21c09f7b0 con 0x55f332e069e0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb237199740 10 client.15075 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb237199740 1 -- 172.21.15.92:0/2146470559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f332b496b0 con 0x55f332e069e0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb215ffb640 1 -- 172.21.15.92:0/2146470559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb228005040 con 0x55f332e069e0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb215ffb640 1 -- 172.21.15.92:0/2146470559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb21c09a520 con 0x55f332e069e0 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb237199740 10 client.15075 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb237199740 10 client.15075 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:52:22.195 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb237199740 20 client.15075 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb237199740 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x55f332e1d2f0 0x55f332e1e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb237199740 4 client.15075 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb237199740 1 -- 172.21.15.92:0/2146470559 --> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f332d78320 con 0x55f332e1d2f0 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.187+0000 7fb23234f640 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x55f332e1d2f0 0x55f332e1e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.188+0000 7fb215ffb640 10 client.15075 ms_handle_connect on v2:172.21.15.144:6838/380126062 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.188+0000 7fb23234f640 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x55f332e1d2f0 0x55f332e1e760 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7126148931ba58e server_cookie=80a4a1156e069e91 in_seq=0 out_seq=0 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.188+0000 7fb215ffb640 1 -- 172.21.15.92:0/2146470559 <== mds.1 v2:172.21.15.144:6838/380126062 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f332d78320 con 0x55f332e1d2f0 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.188+0000 7fb215ffb640 10 client.15075 handle_command_reply: tid=0 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 client.15075 shutdown 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 -- 172.21.15.92:0/2146470559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 msgr2=0x55f332e10ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e970 tx=0x7fb21c009310 comp rx=0 tx=0).stop 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 -- 172.21.15.92:0/2146470559 shutdown_connections 2024-03-19T01:52:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x55f332e1d2f0 0x55f332e1e760 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 0x55f332e17aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb215ffb640 0 client.15075 ms_handle_reset on v2:172.21.15.144:6838/380126062 2024-03-19T01:52:22.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 --2- 172.21.15.92:0/2146470559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e05ff0 0x55f332e10590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 -- 172.21.15.92:0/2146470559 >> 172.21.15.92:0/2146470559 conn(0x55f332e01560 msgr2=0x55f332e007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 -- 172.21.15.92:0/2146470559 shutdown_connections 2024-03-19T01:52:22.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 1 -- 172.21.15.92:0/2146470559 wait complete. 2024-03-19T01:52:22.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.189+0000 7fb237199740 20 client.15075 trim_cache size 0 max 16384 2024-03-19T01:52:22.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:21 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1220816400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:22.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:21 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2578226651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:22.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:21 smithi159 ceph-mon[67110]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-19T01:52:22.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:21 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3277445244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb237199740 1 Processor -- start 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb237199740 1 -- start start 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb237199740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e05ff0 0x55f332e10590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb237199740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb237199740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e0bb50 0x55f332e17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb237199740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f332e1caa0 con 0x55f332e069e0 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb237199740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f332e1d170 con 0x55f332e0bb50 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb237199740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f332e1d840 con 0x55f332e05ff0 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb231b4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb231b4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37156/0 (socket says 172.21.15.92:37156) 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb231b4e640 1 -- 172.21.15.92:0/95659605 learned_addr learned my addr 172.21.15.92:0/95659605 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb23234f640 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e05ff0 0x55f332e10590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb232b50640 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e0bb50 0x55f332e17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb231b4e640 1 -- 172.21.15.92:0/95659605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e05ff0 msgr2=0x55f332e10590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb231b4e640 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e05ff0 0x55f332e10590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb231b4e640 1 -- 172.21.15.92:0/95659605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e0bb50 msgr2=0x55f332e17a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb231b4e640 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e0bb50 0x55f332e17a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb231b4e640 1 -- 172.21.15.92:0/95659605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f332e022d0 con 0x55f332e069e0 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb232b50640 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e0bb50 0x55f332e17a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.207+0000 7fb231b4e640 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb21c099910 tx=0x7fb21c0960a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8015d7a835f274 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb217fff640 1 -- 172.21.15.92:0/95659605 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0860b0 con 0x55f332e069e0 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb217fff640 1 -- 172.21.15.92:0/95659605 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb21c09bd60 con 0x55f332e069e0 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb217fff640 1 -- 172.21.15.92:0/95659605 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c003fb0 con 0x55f332e069e0 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 -- 172.21.15.92:0/95659605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 msgr2=0x55f332e10ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb21c099910 tx=0x7fb21c0960a0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 -- 172.21.15.92:0/95659605 shutdown_connections 2024-03-19T01:52:22.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e0bb50 0x55f332e17a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e069e0 0x55f332e10ad0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 --2- 172.21.15.92:0/95659605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e05ff0 0x55f332e10590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 -- 172.21.15.92:0/95659605 >> 172.21.15.92:0/95659605 conn(0x55f332e01560 msgr2=0x55f332e01940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 -- 172.21.15.92:0/95659605 shutdown_connections 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 -- 172.21.15.92:0/95659605 wait complete. 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.208+0000 7fb237199740 1 -- start start 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb237199740 1 Processor -- start 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb237199740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e05ff0 0x55f332e13290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb237199740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e069e0 0x55f332e137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb237199740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 0x55f332e167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb237199740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f332e1e5e0 con 0x55f332e05ff0 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb237199740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f332e1df10 con 0x55f332e069e0 2024-03-19T01:52:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb237199740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f332e1ecb0 con 0x55f332e0bb50 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb237199740 10 client.0 _ll_register_callbacks cb 0x7fb21c00ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb23234f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e05ff0 0x55f332e13290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb23234f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e05ff0 0x55f332e13290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37168/0 (socket says 172.21.15.92:37168) 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb23234f640 1 -- 172.21.15.92:0/3938926167 learned_addr learned my addr 172.21.15.92:0/3938926167 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb232b50640 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 0x55f332e167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb231b4e640 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e069e0 0x55f332e137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb23234f640 1 -- 172.21.15.92:0/3938926167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 msgr2=0x55f332e167a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb23234f640 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 0x55f332e167a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb23234f640 1 -- 172.21.15.92:0/3938926167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e069e0 msgr2=0x55f332e137d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb23234f640 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e069e0 0x55f332e137d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb23234f640 1 -- 172.21.15.92:0/3938926167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f332e16d90 con 0x55f332e05ff0 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb23234f640 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e05ff0 0x55f332e13290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb228009900 tx=0x7fb228005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1562770b919b14b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.219 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb215ffb640 1 -- 172.21.15.92:0/3938926167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22802c980 con 0x55f332e05ff0 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb237199740 1 -- 172.21.15.92:0/3938926167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb21c09bd60 con 0x55f332e05ff0 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.209+0000 7fb215ffb640 1 -- 172.21.15.92:0/3938926167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb228050670 con 0x55f332e05ff0 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb215ffb640 1 -- 172.21.15.92:0/3938926167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb228004040 con 0x55f332e05ff0 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb237199740 10 client.15081 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb237199740 1 -- 172.21.15.92:0/3938926167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f332e83120 con 0x55f332e05ff0 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb215ffb640 1 -- 172.21.15.92:0/3938926167 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb21c09bd60 con 0x55f332e05ff0 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb215ffb640 1 -- 172.21.15.92:0/3938926167 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb228034440 con 0x55f332e05ff0 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb237199740 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb237199740 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb237199740 20 client.15081 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb237199740 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x55f332e87730 0x55f332e86440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb237199740 4 client.15081 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:22.220 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb237199740 1 -- 172.21.15.92:0/3938926167 --> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] -- command(tid 0: {"prefix": "status"}) -- 0x55f332d72670 con 0x55f332e87730 2024-03-19T01:52:22.221 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb231b4e640 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x55f332e87730 0x55f332e86440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb215ffb640 10 client.15081 ms_handle_connect on v2:172.21.15.144:6838/380126062 2024-03-19T01:52:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.210+0000 7fb231b4e640 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x55f332e87730 0x55f332e86440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c931c03a9dd9b20 server_cookie=e18095a5fbc2ed6e in_seq=0 out_seq=0 2024-03-19T01:52:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb215ffb640 1 -- 172.21.15.92:0/3938926167 <== mds.1 v2:172.21.15.144:6838/380126062 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f332d72670 con 0x55f332e87730 2024-03-19T01:52:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb215ffb640 10 client.15081 handle_command_reply: tid=0 2024-03-19T01:52:22.222 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 client.15081 shutdown 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 -- 172.21.15.92:0/3938926167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e05ff0 msgr2=0x55f332e13290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e05ff0 0x55f332e13290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb228009900 tx=0x7fb228005b50 comp rx=0 tx=0).stop 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 -- 172.21.15.92:0/3938926167 shutdown_connections 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x55f332e87730 0x55f332e86440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f332e0bb50 0x55f332e167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb215ffb640 0 client.15081 ms_handle_reset on v2:172.21.15.144:6838/380126062 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f332e069e0 0x55f332e137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 --2- 172.21.15.92:0/3938926167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f332e05ff0 0x55f332e13290 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 -- 172.21.15.92:0/3938926167 >> 172.21.15.92:0/3938926167 conn(0x55f332e01560 msgr2=0x55f332e07f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 -- 172.21.15.92:0/3938926167 shutdown_connections 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 1 -- 172.21.15.92:0/3938926167 wait complete. 2024-03-19T01:52:22.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb237199740 20 client.15081 trim_cache size 0 max 16384 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24476, 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 89.228912480000005, 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 97.934063953999996 2024-03-19T01:52:22.224 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 -- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2180a2800 msgr2=0x7fb2180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2180a2800 0x7fb2180a4cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55f332e07240 tx=0x7fb224009500 comp rx=0 tx=0).stop 2024-03-19T01:52:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 -- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c104c70 msgr2=0x7fb22c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c104c70 0x7fb22c1980d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e9a0 tx=0x7fb21c00ee60 comp rx=0 tx=0).stop 2024-03-19T01:52:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 -- 172.21.15.92:0/401978997 shutdown_connections 2024-03-19T01:52:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb2180a2800 0x7fb2180a4cc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb22c1075a0 0x7fb22c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb22c104c70 0x7fb22c1980d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 --2- 172.21.15.92:0/401978997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb22c102340 0x7fb22c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.211+0000 7fb2345b3640 1 -- 172.21.15.92:0/401978997 >> 172.21.15.92:0/401978997 conn(0x7fb22c0fbcb0 msgr2=0x7fb22c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.212+0000 7fb2345b3640 1 -- 172.21.15.92:0/401978997 shutdown_connections 2024-03-19T01:52:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.212+0000 7fb2345b3640 1 -- 172.21.15.92:0/401978997 wait complete. 2024-03-19T01:52:22.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-19T01:52:22.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.329+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/524249097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 msgr2=0x7fdbf8104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.329+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/524249097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 0x7fdbf8104730 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009900 tx=0x7fdbe802bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:22.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.329+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/524249097 shutdown_connections 2024-03-19T01:52:22.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.329+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/524249097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbf81075a0 0x7fdbf8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.329+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/524249097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf8104c70 0x7fdbf8107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.329+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/524249097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 0x7fdbf8104730 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.329+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/524249097 >> 172.21.15.92:0/524249097 conn(0x7fdbf80fbcb0 msgr2=0x7fdbf80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/524249097 shutdown_connections 2024-03-19T01:52:22.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/524249097 wait complete. 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 Processor -- start 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 -- start start 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 0x7fdbf8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbf8104c70 0x7fdbf81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf81075a0 0x7fdbf819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810d4b0 con 0x7fdbf8102340 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810d330 con 0x7fdbf81075a0 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.330+0000 7fdbfd3e6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810d630 con 0x7fdbf8104c70 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 0x7fdbf8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 0x7fdbf8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37192/0 (socket says 172.21.15.92:37192) 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/1202382479 learned_addr learned my addr 172.21.15.92:0/1202382479 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbf8104c70 0x7fdbf81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf81075a0 0x7fdbf819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/1202382479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbf8104c70 msgr2=0x7fdbf81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbf8104c70 0x7fdbf81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/1202382479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf81075a0 msgr2=0x7fdbf819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf81075a0 0x7fdbf819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/1202382479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf819f7a0 con 0x7fdbf8102340 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf81075a0 0x7fdbf819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 0x7fdbf8197b80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009230 tx=0x7fdbe8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbdffff640 1 -- 172.21.15.92:0/1202382479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8041070 con 0x7fdbf8102340 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/1202382479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf819fa30 con 0x7fdbf8102340 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbdffff640 1 -- 172.21.15.92:0/1202382479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe80044d0 con 0x7fdbf8102340 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbdffff640 1 -- 172.21.15.92:0/1202382479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8034440 con 0x7fdbf8102340 2024-03-19T01:52:22.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.331+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/1202382479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf819ff20 con 0x7fdbf8102340 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.333+0000 7fdbdffff640 1 -- 172.21.15.92:0/1202382479 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fdbe8044070 con 0x7fdbf8102340 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbfffce740 1 Processor -- start 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbfffce740 1 -- start start 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a08e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08e9e0 0x55b12a0934e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 0x55b12a095fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbfffce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b129ff2de0 con 0x55b12a08dff0 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbfffce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b129f9bc80 con 0x55b12a093b50 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbfffce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b129feee80 con 0x55b12a08e9e0 2024-03-19T01:52:22.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a08e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a08e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37198/0 (socket says 172.21.15.92:37198) 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf77fe640 1 -- 172.21.15.92:0/2971597077 learned_addr learned my addr 172.21.15.92:0/2971597077 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08e9e0 0x55b12a0934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 0x55b12a095fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbdffff640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdbe00a2800 0x7fdbe00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf77fe640 1 -- 172.21.15.92:0/2971597077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08e9e0 msgr2=0x55b12a0934e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08e9e0 0x55b12a0934e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf77fe640 1 -- 172.21.15.92:0/2971597077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 msgr2=0x55b12a095fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 0x55b12a095fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf77fe640 1 -- 172.21.15.92:0/2971597077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b129fadfb0 con 0x55b12a08dff0 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.334+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08e9e0 0x55b12a0934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.335+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdbe00a2800 0x7fdbe00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.335+0000 7fdbdffff640 1 -- 172.21.15.92:0/1202382479 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdbe80f0730 con 0x7fdbf8102340 2024-03-19T01:52:22.342 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.335+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a08e410 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009fd0 tx=0x7fdbf0010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ca4b3292f53b15 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.335+0000 7fdbdd7fa640 1 -- 172.21.15.92:0/2971597077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0019070 con 0x55b12a08dff0 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.335+0000 7fdbdd7fa640 1 -- 172.21.15.92:0/2971597077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf0022070 con 0x55b12a08dff0 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.335+0000 7fdbdd7fa640 1 -- 172.21.15.92:0/2971597077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0011c40 con 0x55b12a08dff0 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 -- 172.21.15.92:0/2971597077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 msgr2=0x55b12a08e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdbe00a2800 0x7fdbe00a4cc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0021000 tx=0x7fdbf001f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a08e410 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009fd0 tx=0x7fdbf0010a00 comp rx=0 tx=0).stop 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 -- 172.21.15.92:0/2971597077 shutdown_connections 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 0x55b12a095fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08e9e0 0x55b12a0934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 --2- 172.21.15.92:0/2971597077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a08e410 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 -- 172.21.15.92:0/2971597077 >> 172.21.15.92:0/2971597077 conn(0x55b12a089560 msgr2=0x55b12a089960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 -- 172.21.15.92:0/2971597077 shutdown_connections 2024-03-19T01:52:22.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 -- 172.21.15.92:0/2971597077 wait complete. 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 -- start start 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 Processor -- start 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08dff0 0x55b12a098560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08e9e0 0x55b12a098aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 0x55b12a09fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b12a096b00 con 0x55b12a08e9e0 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b129faefc0 con 0x55b12a093b50 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b12a096c80 con 0x55b12a08dff0 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbfffce740 10 client.0 _ll_register_callbacks cb 0x55b129feaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbf67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08e9e0 0x55b12a098aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbf67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08e9e0 0x55b12a098aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37212/0 (socket says 172.21.15.92:37212) 2024-03-19T01:52:22.344 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbf67fc640 1 -- 172.21.15.92:0/621037933 learned_addr learned my addr 172.21.15.92:0/621037933 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08dff0 0x55b12a098560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.336+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 0x55b12a09fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbf67fc640 1 -- 172.21.15.92:0/621037933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08dff0 msgr2=0x55b12a098560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08dff0 0x55b12a098560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbf67fc640 1 -- 172.21.15.92:0/621037933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 msgr2=0x55b12a09fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 0x55b12a09fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbf67fc640 1 -- 172.21.15.92:0/621037933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b129d072d0 con 0x55b12a08e9e0 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08e9e0 0x55b12a098aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00bfd0 tx=0x7fdbec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b471924449350e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbdaff5640 1 -- 172.21.15.92:0/621037933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec004020 con 0x55b12a08e9e0 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbdaff5640 1 -- 172.21.15.92:0/621037933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdbec0041c0 con 0x55b12a08e9e0 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbfffce740 1 -- 172.21.15.92:0/621037933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf0022070 con 0x55b12a08e9e0 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbdaff5640 1 -- 172.21.15.92:0/621037933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdbec004400 con 0x55b12a08e9e0 2024-03-19T01:52:22.345 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbfffce740 10 client.15093 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbfffce740 1 -- 172.21.15.92:0/621037933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b129dd16b0 con 0x55b12a08e9e0 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbdaff5640 1 -- 172.21.15.92:0/621037933 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf0022070 con 0x55b12a08e9e0 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.337+0000 7fdbdaff5640 1 -- 172.21.15.92:0/621037933 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdbec0039c0 con 0x55b12a08e9e0 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.338+0000 7fdbfffce740 10 client.15093 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.338+0000 7fdbfffce740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.338+0000 7fdbfffce740 20 client.15093 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.338+0000 7fdbfffce740 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b12a0a5810 0x55b12a0a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.338+0000 7fdbfffce740 4 client.15093 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.338+0000 7fdbfffce740 1 -- 172.21.15.92:0/621037933 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b12a000320 con 0x55b12a0a5810 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.338+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b12a0a5810 0x55b12a0a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.338+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b12a0a5810 0x55b12a0a6c80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee031a8663b6bc8 server_cookie=739c1ca2cdc96ef3 in_seq=0 out_seq=0 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.338+0000 7fdbdaff5640 10 client.15093 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbdaff5640 1 -- 172.21.15.92:0/621037933 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b12a000320 con 0x55b12a0a5810 2024-03-19T01:52:22.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbdaff5640 10 client.15093 handle_command_reply: tid=0 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 client.15093 shutdown 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 -- 172.21.15.92:0/621037933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08e9e0 msgr2=0x55b12a098aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08e9e0 0x55b12a098aa0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00bfd0 tx=0x7fdbec011810 comp rx=0 tx=0).stop 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 -- 172.21.15.92:0/621037933 shutdown_connections 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b12a0a5810 0x55b12a0a6c80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a093b50 0x55b12a09fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08e9e0 0x55b12a098aa0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbdaff5640 0 client.15093 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 --2- 172.21.15.92:0/621037933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a08dff0 0x55b12a098560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 -- 172.21.15.92:0/621037933 >> 172.21.15.92:0/621037933 conn(0x55b12a089560 msgr2=0x55b12a0887e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 -- 172.21.15.92:0/621037933 shutdown_connections 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 1 -- 172.21.15.92:0/621037933 wait complete. 2024-03-19T01:52:22.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.339+0000 7fdbfffce740 20 client.15093 trim_cache size 0 max 16384 2024-03-19T01:52:22.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbfffce740 1 Processor -- start 2024-03-19T01:52:22.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbfffce740 1 -- start start 2024-03-19T01:52:22.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a089940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 0x55b12a09faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 0x55b12a097020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbfffce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b12a0a4fc0 con 0x55b12a08dff0 2024-03-19T01:52:22.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbfffce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b12a0a5690 con 0x55b12a08e9e0 2024-03-19T01:52:22.364 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbfffce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b12a0a5d60 con 0x55b12a093b50 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a089940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a089940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37222/0 (socket says 172.21.15.92:37222) 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/1367803783 learned_addr learned my addr 172.21.15.92:0/1367803783 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 0x55b12a09faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 0x55b12a097020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/1367803783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 msgr2=0x55b12a097020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 0x55b12a097020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/1367803783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 msgr2=0x55b12a09faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 0x55b12a09faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/1367803783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12a08a2d0 con 0x55b12a08dff0 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 0x55b12a09faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 0x55b12a097020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:22.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.357+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a089940 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80b9a50 tx=0x7fdbe80f6530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dd1b03f6bd333c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbdcff9640 1 -- 172.21.15.92:0/1367803783 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe804b040 con 0x55b12a08dff0 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbdcff9640 1 -- 172.21.15.92:0/1367803783 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe80bd780 con 0x55b12a08dff0 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbdcff9640 1 -- 172.21.15.92:0/1367803783 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80c5820 con 0x55b12a08dff0 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 -- 172.21.15.92:0/1367803783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 msgr2=0x55b12a089940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a089940 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80b9a50 tx=0x7fdbe80f6530 comp rx=0 tx=0).stop 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 -- 172.21.15.92:0/1367803783 shutdown_connections 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 0x55b12a097020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 0x55b12a09faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 --2- 172.21.15.92:0/1367803783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a089940 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 -- 172.21.15.92:0/1367803783 >> 172.21.15.92:0/1367803783 conn(0x55b12a089560 msgr2=0x55b12a0a4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 -- 172.21.15.92:0/1367803783 shutdown_connections 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 -- 172.21.15.92:0/1367803783 wait complete. 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 -- start start 2024-03-19T01:52:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.358+0000 7fdbfffce740 1 Processor -- start 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a09aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 0x55b12a09b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbfffce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 0x55b12a09e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbfffce740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b12a0a6b00 con 0x55b12a08dff0 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbfffce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b12a0a6430 con 0x55b12a08e9e0 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbfffce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b12a0a71d0 con 0x55b12a093b50 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbfffce740 10 client.0 _ll_register_callbacks cb 0x7fdbec00bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a09aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a09aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37232/0 (socket says 172.21.15.92:37232) 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/3526360718 learned_addr learned my addr 172.21.15.92:0/3526360718 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 0x55b12a09b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf77fe640 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 0x55b12a09e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/3526360718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 msgr2=0x55b12a09e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 0x55b12a09e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/3526360718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 msgr2=0x55b12a09b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 0x55b12a09b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf6ffd640 1 -- 172.21.15.92:0/3526360718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12a09eb40 con 0x55b12a08dff0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbf6ffd640 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a09aff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdbe802c7b0 tx=0x7fdbe8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a37a26eac44d94 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbdaff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbdaff5640 1 -- 172.21.15.92:0/3526360718 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe804b040 con 0x55b12a08dff0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbdaff5640 1 -- 172.21.15.92:0/3526360718 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdbe80b96f0 con 0x55b12a08dff0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.359+0000 7fdbfffce740 1 -- 172.21.15.92:0/3526360718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe80bd780 con 0x55b12a08dff0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbdaff5640 1 -- 172.21.15.92:0/3526360718 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdbe80bcbc0 con 0x55b12a08dff0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbfffce740 10 client.15099 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbfffce740 1 -- 172.21.15.92:0/3526360718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b12a09fe40 con 0x55b12a08dff0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbdaff5640 1 -- 172.21.15.92:0/3526360718 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe80bd780 con 0x55b12a08dff0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbdaff5640 1 -- 172.21.15.92:0/3526360718 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdbe80c5420 con 0x55b12a08dff0 2024-03-19T01:52:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbfffce740 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbfffce740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbfffce740 20 client.15099 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbfffce740 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b12a10fd70 0x55b12a10ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbfffce740 4 client.15099 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbfffce740 1 -- 172.21.15.92:0/3526360718 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "status"}) -- 0x55b129ffa670 con 0x55b12a10fd70 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b12a10fd70 0x55b12a10ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbdaff5640 10 client.15099 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.360+0000 7fdbf67fc640 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b12a10fd70 0x55b12a10ea80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9407189beba7547a server_cookie=443cf512eb089b70 in_seq=0 out_seq=0 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbdaff5640 1 -- 172.21.15.92:0/3526360718 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b129ffa670 con 0x55b12a10fd70 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbdaff5640 10 client.15099 handle_command_reply: tid=0 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 client.15099 shutdown 2024-03-19T01:52:22.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 -- 172.21.15.92:0/3526360718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 msgr2=0x55b12a09aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a09aff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdbe802c7b0 tx=0x7fdbe8000bc0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 -- 172.21.15.92:0/3526360718 shutdown_connections 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b12a10fd70 0x55b12a10ea80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b12a093b50 0x55b12a09e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbdaff5640 0 client.15099 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b12a08e9e0 0x55b12a09b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 --2- 172.21.15.92:0/3526360718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b12a08dff0 0x55b12a09aff0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 -- 172.21.15.92:0/3526360718 >> 172.21.15.92:0/3526360718 conn(0x55b12a089560 msgr2=0x55b12a0a4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 -- 172.21.15.92:0/3526360718 shutdown_connections 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 1 -- 172.21.15.92:0/3526360718 wait complete. 2024-03-19T01:52:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfffce740 20 client.15099 trim_cache size 0 max 16384 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24493, 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 91.828484435999997, 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 106.787664792 2024-03-19T01:52:22.371 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdbe00a2800 msgr2=0x7fdbe00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdbe00a2800 0x7fdbe00a4cc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0021000 tx=0x7fdbf001f000 comp rx=0 tx=0).stop 2024-03-19T01:52:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 msgr2=0x7fdbf8197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 0x7fdbf8197b80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009230 tx=0x7fdbe8005d50 comp rx=0 tx=0).stop 2024-03-19T01:52:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/1202382479 shutdown_connections 2024-03-19T01:52:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf81075a0 0x7fdbf819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdbe00a2800 0x7fdbe00a4cc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbf8104c70 0x7fdbf81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 --2- 172.21.15.92:0/1202382479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf8102340 0x7fdbf8197b80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.361+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/1202382479 >> 172.21.15.92:0/1202382479 conn(0x7fdbf80fbcb0 msgr2=0x7fdbf8102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.362+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/1202382479 shutdown_connections 2024-03-19T01:52:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:22.362+0000 7fdbfd3e6640 1 -- 172.21.15.92:0/1202382479 wait complete. 2024-03-19T01:52:22.374 INFO:tasks.cephadm:Setup complete, yielding 2024-03-19T01:52:22.374 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-19T01:52:22.384 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-03-19T01:52:22.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-19T01:52:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.458+0000 7f47b10cf700 1 -- 172.21.15.92:0/2702000062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac102de0 msgr2=0x7f47ac109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.458+0000 7f47b10cf700 1 --2- 172.21.15.92:0/2702000062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac102de0 0x7f47ac109690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f47a400b0d0 tx=0x7f47a400b3e0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.458+0000 7f47b10cf700 1 -- 172.21.15.92:0/2702000062 shutdown_connections 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.458+0000 7f47b10cf700 1 --2- 172.21.15.92:0/2702000062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac102de0 0x7f47ac109690 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.458+0000 7f47b10cf700 1 --2- 172.21.15.92:0/2702000062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac101240 0x7f47ac101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.458+0000 7f47b10cf700 1 --2- 172.21.15.92:0/2702000062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102440 0x7f47ac1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.458+0000 7f47b10cf700 1 -- 172.21.15.92:0/2702000062 >> 172.21.15.92:0/2702000062 conn(0x7f47ac0fc7e0 msgr2=0x7f47ac0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.458+0000 7f47b10cf700 1 -- 172.21.15.92:0/2702000062 shutdown_connections 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.458+0000 7f47b10cf700 1 -- 172.21.15.92:0/2702000062 wait complete. 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47b10cf700 1 Processor -- start 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47b10cf700 1 -- start start 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47b10cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac101240 0x7f47ac06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47b10cf700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102440 0x7f47ac19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47b10cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac102de0 0x7f47ac19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47b10cf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac10ce20 con 0x7f47ac101240 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47b10cf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac10cca0 con 0x7f47ac102de0 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47b10cf700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac10cfa0 con 0x7f47ac102440 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47aad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac101240 0x7f47ac06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47aad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac101240 0x7f47ac06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53002/0 (socket says 172.21.15.92:53002) 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47aad9d700 1 -- 172.21.15.92:0/1696252868 learned_addr learned my addr 172.21.15.92:0/1696252868 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47aa59c700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102440 0x7f47ac19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.459+0000 7f47ab59e700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac102de0 0x7f47ac19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f47aad9d700 1 -- 172.21.15.92:0/1696252868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102440 msgr2=0x7f47ac19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f47aad9d700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102440 0x7f47ac19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f47aad9d700 1 -- 172.21.15.92:0/1696252868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac102de0 msgr2=0x7f47ac19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f47aad9d700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac102de0 0x7f47ac19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f47aad9d700 1 -- 172.21.15.92:0/1696252868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ac1a0190 con 0x7f47ac101240 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f47aad9d700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac101240 0x7f47ac06ab40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e970 tx=0x7f479c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f4793fff700 1 -- 172.21.15.92:0/1696252868 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c004220 con 0x7f47ac101240 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ac1a0480 con 0x7f47ac101240 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f4793fff700 1 -- 172.21.15.92:0/1696252868 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f479c0043c0 con 0x7f47ac101240 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f4793fff700 1 -- 172.21.15.92:0/1696252868 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c003dc0 con 0x7f47ac101240 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.460+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ac1a0a10 con 0x7f47ac101240 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.461+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ac068fd0 con 0x7f47ac101240 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.463+0000 7f4793fff700 1 -- 172.21.15.92:0/1696252868 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f479c00c7e0 con 0x7f47ac101240 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.463+0000 7f4793fff700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f47940a2890 0x7f47940a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.463+0000 7f4793fff700 1 -- 172.21.15.92:0/1696252868 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f479c0cecd0 con 0x7f47ac101240 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.463+0000 7f47aa59c700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f47940a2890 0x7f47940a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.464+0000 7f47aa59c700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f47940a2890 0x7f47940a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f47ac06b0a0 tx=0x7f47a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:24.468 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.467+0000 7f4793fff700 1 -- 172.21.15.92:0/1696252868 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f479c09be10 con 0x7f47ac101240 2024-03-19T01:52:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.689+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f47ac062750 con 0x7f47940a2890 2024-03-19T01:52:24.689 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:24 smithi092 ceph-mon[62574]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-19T01:52:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.690+0000 7f4793fff700 1 -- 172.21.15.92:0/1696252868 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f47ac062750 con 0x7f47940a2890 2024-03-19T01:52:24.690 INFO:teuthology.orchestra.run.smithi092.stdout:Backend: cephadm 2024-03-19T01:52:24.690 INFO:teuthology.orchestra.run.smithi092.stdout:Available: Yes 2024-03-19T01:52:24.690 INFO:teuthology.orchestra.run.smithi092.stdout:Paused: No 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f47940a2890 msgr2=0x7f47940a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f47940a2890 0x7f47940a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f47ac06b0a0 tx=0x7f47a0009260 comp rx=0 tx=0).stop 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac101240 msgr2=0x7f47ac06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac101240 0x7f47ac06ab40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e970 tx=0x7f479c00ec80 comp rx=0 tx=0).stop 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 shutdown_connections 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f47940a2890 0x7f47940a4d50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ac102de0 0x7f47ac19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac101240 0x7f47ac06ab40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 --2- 172.21.15.92:0/1696252868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102440 0x7f47ac19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 >> 172.21.15.92:0/1696252868 conn(0x7f47ac0fc7e0 msgr2=0x7f47ac105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 shutdown_connections 2024-03-19T01:52:24.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:24.693+0000 7f47b10cf700 1 -- 172.21.15.92:0/1696252868 wait complete. 2024-03-19T01:52:24.799 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:24 smithi144 ceph-mon[67199]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-19T01:52:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:24 smithi159 ceph-mon[67110]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-19T01:52:25.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-19T01:52:25.691 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:26.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:26 smithi092 ceph-mon[62574]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:26.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:26 smithi092 ceph-mon[62574]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:26.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:26 smithi144 ceph-mon[67199]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:26.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:26 smithi144 ceph-mon[67199]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:26.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:26 smithi159 ceph-mon[67110]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:26.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:26 smithi159 ceph-mon[67110]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 -- 172.21.15.92:0/2970465136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff090108020 msgr2=0x7ff09010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 --2- 172.21.15.92:0/2970465136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff090108020 0x7ff09010a490 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b0a0 tx=0x7ff08800b3b0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 -- 172.21.15.92:0/2970465136 shutdown_connections 2024-03-19T01:52:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 --2- 172.21.15.92:0/2970465136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff090108020 0x7ff09010a490 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 --2- 172.21.15.92:0/2970465136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff090102dc0 0x7ff0901051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 --2- 172.21.15.92:0/2970465136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0901056f0 0x7ff090107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 -- 172.21.15.92:0/2970465136 >> 172.21.15.92:0/2970465136 conn(0x7ff0900fc7d0 msgr2=0x7ff0900fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 -- 172.21.15.92:0/2970465136 shutdown_connections 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 -- 172.21.15.92:0/2970465136 wait complete. 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 Processor -- start 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.578+0000 7ff096571700 1 -- start start 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff096571700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff090102dc0 0x7ff0901984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff096571700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0901056f0 0x7ff0901989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff096571700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff090108020 0x7ff09019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff096571700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff09010dee0 con 0x7ff0901056f0 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff096571700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff09010dd60 con 0x7ff090102dc0 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff096571700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff09010e060 con 0x7ff090108020 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff08f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0901056f0 0x7ff0901989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:27.581 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff08f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0901056f0 0x7ff0901989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53022/0 (socket says 172.21.15.92:53022) 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff08f7fe700 1 -- 172.21.15.92:0/1910352550 learned_addr learned my addr 172.21.15.92:0/1910352550 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff094b0e700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff090108020 0x7ff09019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff08ffff700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff090102dc0 0x7ff0901984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff08f7fe700 1 -- 172.21.15.92:0/1910352550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff090108020 msgr2=0x7ff09019fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff08f7fe700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff090108020 0x7ff09019fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff08f7fe700 1 -- 172.21.15.92:0/1910352550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff090102dc0 msgr2=0x7ff0901984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff08f7fe700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff090102dc0 0x7ff0901984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.579+0000 7ff08f7fe700 1 -- 172.21.15.92:0/1910352550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0901a0100 con 0x7ff0901056f0 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.580+0000 7ff08f7fe700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0901056f0 0x7ff0901989e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff08400e9d0 tx=0x7ff08400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.580+0000 7ff08d7fa700 1 -- 172.21.15.92:0/1910352550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400cbd0 con 0x7ff0901056f0 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.580+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0901a03f0 con 0x7ff0901056f0 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.580+0000 7ff08d7fa700 1 -- 172.21.15.92:0/1910352550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08400ee70 con 0x7ff0901056f0 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.580+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901a0980 con 0x7ff0901056f0 2024-03-19T01:52:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.580+0000 7ff08d7fa700 1 -- 172.21.15.92:0/1910352550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084018680 con 0x7ff0901056f0 2024-03-19T01:52:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.581+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff090068fd0 con 0x7ff0901056f0 2024-03-19T01:52:27.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.582+0000 7ff08d7fa700 1 -- 172.21.15.92:0/1910352550 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff0840188e0 con 0x7ff0901056f0 2024-03-19T01:52:27.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.582+0000 7ff08d7fa700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff0780a2890 0x7ff0780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:27.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.582+0000 7ff08ffff700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff0780a2890 0x7ff0780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:27.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.582+0000 7ff08d7fa700 1 -- 172.21.15.92:0/1910352550 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff0840cec40 con 0x7ff0901056f0 2024-03-19T01:52:27.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.583+0000 7ff08ffff700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff0780a2890 0x7ff0780a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff080009680 tx=0x7ff080009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.591+0000 7ff08d7fa700 1 -- 172.21.15.92:0/1910352550 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff08409bcf0 con 0x7ff0901056f0 2024-03-19T01:52:27.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.798+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff090062750 con 0x7ff0780a2890 2024-03-19T01:52:27.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.812+0000 7ff08d7fa700 1 -- 172.21.15.92:0/1910352550 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7ff090062750 con 0x7ff0780a2890 2024-03-19T01:52:27.812 INFO:teuthology.orchestra.run.smithi092.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-19T01:52:27.812 INFO:teuthology.orchestra.run.smithi092.stdout:mds.a smithi092 running (106s) 19s ago 106s 17.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 93b782d6d8b8 2024-03-19T01:52:27.812 INFO:teuthology.orchestra.run.smithi092.stdout:mds.b smithi144 running (103s) 37s ago 103s 17.5M - 19.0.0-2206-gb6ae2491 2312c8e42d00 7fdf2e218f60 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.c smithi159 running (118s) 37s ago 117s 17.4M - 19.0.0-2206-gb6ae2491 2312c8e42d00 75c7a3f9696f 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.d smithi092 running (2m) 19s ago 2m 15.3M - 19.0.0-2206-gb6ae2491 2312c8e42d00 1e4567c99fe6 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.e smithi144 running (2m) 37s ago 2m 16.5M - 19.0.0-2206-gb6ae2491 2312c8e42d00 05838348eb27 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.f smithi159 running (2m) 37s ago 2m 17.5M - 19.0.0-2206-gb6ae2491 2312c8e42d00 dc3a13d7a5c0 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.g smithi092 running (109s) 19s ago 109s 16.2M - 19.0.0-2206-gb6ae2491 2312c8e42d00 7136f2cd665d 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.h smithi144 running (115s) 37s ago 115s 17.7M - 19.0.0-2206-gb6ae2491 2312c8e42d00 b768b4be8e7b 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.i smithi159 running (112s) 37s ago 112s 26.1M - 19.0.0-2206-gb6ae2491 2312c8e42d00 37fa8cc2c9bf 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.j smithi092 running (2m) 19s ago 2m 16.1M - 19.0.0-2206-gb6ae2491 2312c8e42d00 91ba3f745ff1 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.k smithi144 running (2m) 37s ago 2m 19.1M - 19.0.0-2206-gb6ae2491 2312c8e42d00 e735324ea198 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mds.l smithi159 running (2m) 37s ago 2m 16.9M - 19.0.0-2206-gb6ae2491 2312c8e42d00 9f5dc9a82c9b 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.x smithi092 *:9283,8765 running (9m) 19s ago 9m 513M - 19.0.0-2206-gb6ae2491 2312c8e42d00 56cbeab0cd4c 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.y smithi144 *:8443,8765 running (7m) 37s ago 7m 451M - 19.0.0-2206-gb6ae2491 2312c8e42d00 fbebc8f9da79 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mgr.z smithi159 *:8443,8765 running (7m) 37s ago 7m 453M - 19.0.0-2206-gb6ae2491 2312c8e42d00 259c300c508e 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a smithi092 running (9m) 19s ago 9m 52.0M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 9b3aea2ff4cd 2024-03-19T01:52:27.813 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b smithi144 running (7m) 37s ago 7m 41.0M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 63cf629954ae 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c smithi159 running (8m) 37s ago 8m 39.0M 2048M 19.0.0-2206-gb6ae2491 2312c8e42d00 31e125f9f5c4 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 smithi092 running (7m) 19s ago 7m 55.8M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 e8aa00aea4b2 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 smithi144 running (6m) 37s ago 6m 46.4M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 17d3c096fe1d 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 smithi159 running (6m) 37s ago 6m 59.2M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 b15129fcee2f 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 smithi092 running (6m) 19s ago 6m 43.6M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 22301136b211 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 smithi144 running (5m) 37s ago 5m 45.1M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 6f15b84e1e26 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 smithi159 running (5m) 37s ago 5m 44.5M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 47f662b193e1 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 smithi092 running (4m) 19s ago 4m 52.8M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 621eab5e8a61 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 smithi144 running (4m) 37s ago 4m 54.4M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 69da6b9bf2ef 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.8 smithi159 running (3m) 37s ago 3m 43.6M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 22c441205f06 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.9 smithi092 running (3m) 19s ago 3m 44.3M 4272M 19.0.0-2206-gb6ae2491 2312c8e42d00 faf050e74e25 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.10 smithi144 running (3m) 37s ago 3m 43.1M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 c86852528659 2024-03-19T01:52:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:osd.11 smithi159 running (2m) 37s ago 2m 43.7M 4270M 19.0.0-2206-gb6ae2491 2312c8e42d00 f93f0f685ef2 2024-03-19T01:52:27.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff0780a2890 msgr2=0x7ff0780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:27.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff0780a2890 0x7ff0780a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff080009680 tx=0x7ff080009260 comp rx=0 tx=0).stop 2024-03-19T01:52:27.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0901056f0 msgr2=0x7ff0901989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:27.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0901056f0 0x7ff0901989e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff08400e9d0 tx=0x7ff08400ece0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.815 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 shutdown_connections 2024-03-19T01:52:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff0780a2890 0x7ff0780a4d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff090108020 0x7ff09019fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff090102dc0 0x7ff0901984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 --2- 172.21.15.92:0/1910352550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0901056f0 0x7ff0901989e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 >> 172.21.15.92:0/1910352550 conn(0x7ff0900fc7d0 msgr2=0x7ff090103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 shutdown_connections 2024-03-19T01:52:27.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:27.814+0000 7ff096571700 1 -- 172.21.15.92:0/1910352550 wait complete. 2024-03-19T01:52:28.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-19T01:52:28.779 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:28.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:28 smithi092 ceph-mon[62574]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:28 smithi144 ceph-mon[67199]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:28 smithi159 ceph-mon[67110]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:29.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:29 smithi092 ceph-mon[62574]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:29.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:29 smithi144 ceph-mon[67199]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:29.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:29 smithi159 ceph-mon[67110]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.478+0000 7fbd07605700 1 -- 172.21.15.92:0/1267721451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd00102440 msgr2=0x7fbd001028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.478+0000 7fbd07605700 1 --2- 172.21.15.92:0/1267721451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd00102440 0x7fbd001028a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009990 tx=0x7fbcf0009ca0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 -- 172.21.15.92:0/1267721451 shutdown_connections 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 --2- 172.21.15.92:0/1267721451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd00102de0 0x7fbd00109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 --2- 172.21.15.92:0/1267721451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd00101240 0x7fbd00101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 --2- 172.21.15.92:0/1267721451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd00102440 0x7fbd001028a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 -- 172.21.15.92:0/1267721451 >> 172.21.15.92:0/1267721451 conn(0x7fbd000fc7e0 msgr2=0x7fbd000fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 -- 172.21.15.92:0/1267721451 shutdown_connections 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 -- 172.21.15.92:0/1267721451 wait complete. 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 Processor -- start 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 -- start start 2024-03-19T01:52:30.481 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd00101240 0x7fbd00198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd00102440 0x7fbd00198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd00102de0 0x7fbd0019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0010ce50 con 0x7fbd00101240 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0010ccd0 con 0x7fbd00102440 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.479+0000 7fbd07605700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0010cfd0 con 0x7fbd00102de0 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd053a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd00101240 0x7fbd00198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd053a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd00101240 0x7fbd00198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53040/0 (socket says 172.21.15.92:53040) 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd053a1700 1 -- 172.21.15.92:0/1842402504 learned_addr learned my addr 172.21.15.92:0/1842402504 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd05ba2700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd00102de0 0x7fbd0019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd04ba0700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd00102440 0x7fbd00198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd053a1700 1 -- 172.21.15.92:0/1842402504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd00102de0 msgr2=0x7fbd0019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd053a1700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd00102de0 0x7fbd0019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd053a1700 1 -- 172.21.15.92:0/1842402504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd00102440 msgr2=0x7fbd00198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:30.482 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd053a1700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd00102440 0x7fbd00198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd053a1700 1 -- 172.21.15.92:0/1842402504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd001a0240 con 0x7fbd00101240 2024-03-19T01:52:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd053a1700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd00101240 0x7fbd00198630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00ea80 tx=0x7fbcfc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbcf67fc700 1 -- 172.21.15.92:0/1842402504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00cbd0 con 0x7fbd00101240 2024-03-19T01:52:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd001a0530 con 0x7fbd00101240 2024-03-19T01:52:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd001a0ac0 con 0x7fbd00101240 2024-03-19T01:52:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbcf67fc700 1 -- 172.21.15.92:0/1842402504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcfc00cd70 con 0x7fbd00101240 2024-03-19T01:52:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.480+0000 7fbcf67fc700 1 -- 172.21.15.92:0/1842402504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc018760 con 0x7fbd00101240 2024-03-19T01:52:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.481+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce4005360 con 0x7fbd00101240 2024-03-19T01:52:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.483+0000 7fbcf67fc700 1 -- 172.21.15.92:0/1842402504 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fbcfc01f020 con 0x7fbd00101240 2024-03-19T01:52:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.484+0000 7fbcf67fc700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbcec0a2910 0x7fbcec0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.484+0000 7fbd04ba0700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbcec0a2910 0x7fbcec0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.484+0000 7fbcf67fc700 1 -- 172.21.15.92:0/1842402504 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbcfc0cef10 con 0x7fbd00101240 2024-03-19T01:52:30.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.484+0000 7fbd04ba0700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbcec0a2910 0x7fbcec0a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b2e0 tx=0x7fbcf0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:30.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.487+0000 7fbcf67fc700 1 -- 172.21.15.92:0/1842402504 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbcfc09bf60 con 0x7fbd00101240 2024-03-19T01:52:30.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.696+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fbce4000bf0 con 0x7fbcec0a2910 2024-03-19T01:52:30.702 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.702+0000 7fbcf67fc700 1 -- 172.21.15.92:0/1842402504 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fbce4000bf0 con 0x7fbcec0a2910 2024-03-19T01:52:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-19T01:52:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:mds.all 12/12 40s ago 2m smithi092=a;smithi092=d;smithi092=g;smithi092=j;smithi144=b;smithi144=e;smithi144=h;smithi144=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-03-19T01:52:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:mgr 3/3 40s ago 7m smithi092=x;smithi144=y;smithi159=z;count:3 2024-03-19T01:52:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:mon 3/3 40s ago 8m smithi092:172.21.15.92=a;smithi144:172.21.15.144=b;smithi159:172.21.15.159=c;count:3 2024-03-19T01:52:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:osd 12 40s ago - 2024-03-19T01:52:30.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbcec0a2910 msgr2=0x7fbcec0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:30.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbcec0a2910 0x7fbcec0a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000b2e0 tx=0x7fbcf0005fd0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd00101240 msgr2=0x7fbd00198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:30.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd00101240 0x7fbd00198630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00ea80 tx=0x7fbcfc00ed90 comp rx=0 tx=0).stop 2024-03-19T01:52:30.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 shutdown_connections 2024-03-19T01:52:30.705 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fbcec0a2910 0x7fbcec0a4dd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd00102de0 0x7fbd0019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd00101240 0x7fbd00198630 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 --2- 172.21.15.92:0/1842402504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd00102440 0x7fbd00198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:30.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 >> 172.21.15.92:0/1842402504 conn(0x7fbd000fc7e0 msgr2=0x7fbd000fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:30.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 shutdown_connections 2024-03-19T01:52:30.706 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:30.704+0000 7fbd07605700 1 -- 172.21.15.92:0/1842402504 wait complete. 2024-03-19T01:52:30.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:30 smithi092 ceph-mon[62574]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:30.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:30 smithi144 ceph-mon[67199]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:30 smithi159 ceph-mon[67110]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:31.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-19T01:52:31.760 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:32 smithi092 ceph-mon[62574]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:32.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:32 smithi092 ceph-mon[62574]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:32.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:32 smithi144 ceph-mon[67199]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:32.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:32 smithi144 ceph-mon[67199]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:32 smithi159 ceph-mon[67110]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:32 smithi159 ceph-mon[67110]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:33.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.598+0000 7fe12b497700 1 -- 172.21.15.92:0/629261643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124103ee0 msgr2=0x7fe12410a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:33.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.598+0000 7fe12b497700 1 --2- 172.21.15.92:0/629261643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124103ee0 0x7fe12410a790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00b0d0 tx=0x7fe11c00b3e0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.598+0000 7fe12b497700 1 -- 172.21.15.92:0/629261643 shutdown_connections 2024-03-19T01:52:33.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.598+0000 7fe12b497700 1 --2- 172.21.15.92:0/629261643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124103ee0 0x7fe12410a790 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.598+0000 7fe12b497700 1 --2- 172.21.15.92:0/629261643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe124102340 0x7fe124102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.598+0000 7fe12b497700 1 --2- 172.21.15.92:0/629261643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe124103540 0x7fe1241039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.598+0000 7fe12b497700 1 -- 172.21.15.92:0/629261643 >> 172.21.15.92:0/629261643 conn(0x7fe1240fd8c0 msgr2=0x7fe1240ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:33.600 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.598+0000 7fe12b497700 1 -- 172.21.15.92:0/629261643 shutdown_connections 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.598+0000 7fe12b497700 1 -- 172.21.15.92:0/629261643 wait complete. 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe12b497700 1 Processor -- start 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe12b497700 1 -- start start 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe12b497700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124102340 0x7fe1240760b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe12b497700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe124103540 0x7fe124074700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe12b497700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe124103ee0 0x7fe124074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe12b497700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe12410df30 con 0x7fe124102340 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe12b497700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe12410ddb0 con 0x7fe124103540 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe12b497700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe12410e0b0 con 0x7fe124103ee0 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129233700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124102340 0x7fe1240760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129233700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124102340 0x7fe1240760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53052/0 (socket says 172.21.15.92:53052) 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129233700 1 -- 172.21.15.92:0/1185949914 learned_addr learned my addr 172.21.15.92:0/1185949914 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129a34700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe124103ee0 0x7fe124074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:33.601 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe128a32700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe124103540 0x7fe124074700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129233700 1 -- 172.21.15.92:0/1185949914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe124103ee0 msgr2=0x7fe124074c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129233700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe124103ee0 0x7fe124074c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129233700 1 -- 172.21.15.92:0/1185949914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe124103540 msgr2=0x7fe124074700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129233700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe124103540 0x7fe124074700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129233700 1 -- 172.21.15.92:0/1185949914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1240754f0 con 0x7fe124102340 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.599+0000 7fe129233700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124102340 0x7fe1240760b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe12000e970 tx=0x7fe12000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.600+0000 7fe11a7fc700 1 -- 172.21.15.92:0/1185949914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120004220 con 0x7fe124102340 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.600+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1241a2670 con 0x7fe124102340 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.600+0000 7fe11a7fc700 1 -- 172.21.15.92:0/1185949914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1200043c0 con 0x7fe124102340 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.600+0000 7fe11a7fc700 1 -- 172.21.15.92:0/1185949914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120003dc0 con 0x7fe124102340 2024-03-19T01:52:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.600+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1241a2b60 con 0x7fe124102340 2024-03-19T01:52:33.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.601+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe124068fd0 con 0x7fe124102340 2024-03-19T01:52:33.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.603+0000 7fe11a7fc700 1 -- 172.21.15.92:0/1185949914 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe12000c7e0 con 0x7fe124102340 2024-03-19T01:52:33.603 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.603+0000 7fe11a7fc700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1100a2890 0x7fe1100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:33.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.603+0000 7fe128a32700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1100a2890 0x7fe1100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:33.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.603+0000 7fe11a7fc700 1 -- 172.21.15.92:0/1185949914 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe1200cecd0 con 0x7fe124102340 2024-03-19T01:52:33.604 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.604+0000 7fe128a32700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1100a2890 0x7fe1100a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe114009c60 tx=0x7fe114009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:33.607 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.607+0000 7fe11a7fc700 1 -- 172.21.15.92:0/1185949914 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe12009be10 con 0x7fe124102340 2024-03-19T01:52:33.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.830+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fe124062750 con 0x7fe1100a2890 2024-03-19T01:52:33.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.831+0000 7fe11a7fc700 1 -- 172.21.15.92:0/1185949914 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fe124062750 con 0x7fe1100a2890 2024-03-19T01:52:33.832 INFO:teuthology.orchestra.run.smithi092.stdout:HOST ADDR LABELS STATUS 2024-03-19T01:52:33.832 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092 172.21.15.92 2024-03-19T01:52:33.832 INFO:teuthology.orchestra.run.smithi092.stdout:smithi144 172.21.15.144 2024-03-19T01:52:33.832 INFO:teuthology.orchestra.run.smithi092.stdout:smithi159 172.21.15.159 2024-03-19T01:52:33.832 INFO:teuthology.orchestra.run.smithi092.stdout:3 hosts in cluster 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1100a2890 msgr2=0x7fe1100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1100a2890 0x7fe1100a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe114009c60 tx=0x7fe114009260 comp rx=0 tx=0).stop 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124102340 msgr2=0x7fe1240760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124102340 0x7fe1240760b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe12000e970 tx=0x7fe12000ec80 comp rx=0 tx=0).stop 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 shutdown_connections 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1100a2890 0x7fe1100a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe124103ee0 0x7fe124074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe124102340 0x7fe1240760b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 --2- 172.21.15.92:0/1185949914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe124103540 0x7fe124074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:33.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 >> 172.21.15.92:0/1185949914 conn(0x7fe1240fd8c0 msgr2=0x7fe124106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:33.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.833+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 shutdown_connections 2024-03-19T01:52:33.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:33.834+0000 7fe12b497700 1 -- 172.21.15.92:0/1185949914 wait complete. 2024-03-19T01:52:34.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-19T01:52:34.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:34 smithi092 ceph-mon[62574]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:34 smithi144 ceph-mon[67199]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:34.879 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:34 smithi159 ceph-mon[67110]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:35.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:35 smithi092 ceph-mon[62574]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:35.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:35 smithi144 ceph-mon[67199]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:35.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:35 smithi159 ceph-mon[67110]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:36.664 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:36 smithi092 ceph-mon[62574]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.767+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2582518851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4105700 msgr2=0x7fa4a4107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.767+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2582518851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4105700 0x7fa4a4107af0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa490009990 tx=0x7fa490009ca0 comp rx=0 tx=0).stop 2024-03-19T01:52:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.767+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2582518851 shutdown_connections 2024-03-19T01:52:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.767+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2582518851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a4108030 0x7fa4a410a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.767+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2582518851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4a4102dd0 0x7fa4a41051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.767+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2582518851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4105700 0x7fa4a4107af0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.767+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2582518851 >> 172.21.15.92:0/2582518851 conn(0x7fa4a40fc7e0 msgr2=0x7fa4a40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:36.771 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.767+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2582518851 shutdown_connections 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.768+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2582518851 wait complete. 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.768+0000 7fa4ab07e700 1 Processor -- start 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.768+0000 7fa4ab07e700 1 -- start start 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.768+0000 7fa4ab07e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4102dd0 0x7fa4a41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.768+0000 7fa4ab07e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a4105700 0x7fa4a4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4ab07e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4a4108030 0x7fa4a419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4ab07e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a410dfa0 con 0x7fa4a4102dd0 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4ab07e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a410de20 con 0x7fa4a4105700 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4ab07e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a410e120 con 0x7fa4a4108030 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a8e1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4102dd0 0x7fa4a41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a8e1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4102dd0 0x7fa4a41985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55762/0 (socket says 172.21.15.92:55762) 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a8e1a700 1 -- 172.21.15.92:0/2518876909 learned_addr learned my addr 172.21.15.92:0/2518876909 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a3fff700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a4105700 0x7fa4a4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a961b700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4a4108030 0x7fa4a419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a8e1a700 1 -- 172.21.15.92:0/2518876909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4a4108030 msgr2=0x7fa4a419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a8e1a700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4a4108030 0x7fa4a419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a8e1a700 1 -- 172.21.15.92:0/2518876909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a4105700 msgr2=0x7fa4a4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a8e1a700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a4105700 0x7fa4a4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a8e1a700 1 -- 172.21.15.92:0/2518876909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a4100bc0 con 0x7fa4a4102dd0 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a3fff700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a4105700 0x7fa4a4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a8e1a700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4102dd0 0x7fa4a41985c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa49800e9f0 tx=0x7fa49800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a1ffb700 1 -- 172.21.15.92:0/2518876909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49800cbd0 con 0x7fa4a4102dd0 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a1ffb700 1 -- 172.21.15.92:0/2518876909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa49800ee90 con 0x7fa4a4102dd0 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4a1ffb700 1 -- 172.21.15.92:0/2518876909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4980056c0 con 0x7fa4a4102dd0 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.769+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a4100eb0 con 0x7fa4a4102dd0 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.770+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a41013a0 con 0x7fa4a4102dd0 2024-03-19T01:52:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.771+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a4068fd0 con 0x7fa4a4102dd0 2024-03-19T01:52:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.772+0000 7fa4a1ffb700 1 -- 172.21.15.92:0/2518876909 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fa498004750 con 0x7fa4a4102dd0 2024-03-19T01:52:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.772+0000 7fa4a1ffb700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa4940a2910 0x7fa4940a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.773+0000 7fa4a3fff700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa4940a2910 0x7fa4940a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.773+0000 7fa4a1ffb700 1 -- 172.21.15.92:0/2518876909 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa4980cec10 con 0x7fa4a4102dd0 2024-03-19T01:52:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.773+0000 7fa4a3fff700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa4940a2910 0x7fa4940a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa49000b2e0 tx=0x7fa490005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:36.777+0000 7fa4a1ffb700 1 -- 172.21.15.92:0/2518876909 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa49809bcc0 con 0x7fa4a4102dd0 2024-03-19T01:52:36.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:36 smithi144 ceph-mon[67199]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:36 smithi159 ceph-mon[67110]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:37.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.007+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa4a40fdf70 con 0x7fa4940a2910 2024-03-19T01:52:37.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.010+0000 7fa4a1ffb700 1 -- 172.21.15.92:0/2518876909 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa4a40fdf70 con 0x7fa4940a2910 2024-03-19T01:52:37.011 INFO:teuthology.orchestra.run.smithi092.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-19T01:52:37.011 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-19T01:52:37.011 INFO:teuthology.orchestra.run.smithi092.stdout:smithi144 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050085400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-19T01:52:37.011 INFO:teuthology.orchestra.run.smithi092.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-19T01:52:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa4940a2910 msgr2=0x7fa4940a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa4940a2910 0x7fa4940a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa49000b2e0 tx=0x7fa490005fb0 comp rx=0 tx=0).stop 2024-03-19T01:52:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4102dd0 msgr2=0x7fa4a41985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4102dd0 0x7fa4a41985c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa49800e9f0 tx=0x7fa49800ed00 comp rx=0 tx=0).stop 2024-03-19T01:52:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 shutdown_connections 2024-03-19T01:52:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa4940a2910 0x7fa4940a4dd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:37.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4a4108030 0x7fa4a419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:37.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a4102dd0 0x7fa4a41985c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:37.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 --2- 172.21.15.92:0/2518876909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a4105700 0x7fa4a4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:37.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 >> 172.21.15.92:0/2518876909 conn(0x7fa4a40fc7e0 msgr2=0x7fa4a4104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:37.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 shutdown_connections 2024-03-19T01:52:37.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:37.012+0000 7fa4ab07e700 1 -- 172.21.15.92:0/2518876909 wait complete. 2024-03-19T01:52:37.829 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-19T01:52:37.838 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-03-19T01:52:37.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-19T01:52:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:38.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:38 smithi092 ceph-mon[62574]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:38.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:38 smithi092 ceph-mon[62574]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:38.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:38 smithi144 ceph-mon[67199]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:38.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:38 smithi144 ceph-mon[67199]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:38 smithi159 ceph-mon[67110]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:38 smithi159 ceph-mon[67110]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:39.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 -- 172.21.15.92:0/2892792009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 msgr2=0x7fc36c103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 --2- 172.21.15.92:0/2892792009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 0x7fc36c103ae0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc36000b0a0 tx=0x7fc36000b3b0 comp rx=0 tx=0).stop 2024-03-19T01:52:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 -- 172.21.15.92:0/2892792009 shutdown_connections 2024-03-19T01:52:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 --2- 172.21.15.92:0/2892792009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc36c104020 0x7fc36c10a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 --2- 172.21.15.92:0/2892792009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c100600 0x7fc36c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 --2- 172.21.15.92:0/2892792009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 0x7fc36c103ae0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 -- 172.21.15.92:0/2892792009 >> 172.21.15.92:0/2892792009 conn(0x7fc36c0fbf80 msgr2=0x7fc36c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 -- 172.21.15.92:0/2892792009 shutdown_connections 2024-03-19T01:52:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 -- 172.21.15.92:0/2892792009 wait complete. 2024-03-19T01:52:39.950 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.949+0000 7fc371fde700 1 Processor -- start 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc371fde700 1 -- start start 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc371fde700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c100600 0x7fc36c102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc371fde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 0x7fc36c101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc371fde700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc36c104020 0x7fc36c101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc371fde700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c10df20 con 0x7fc36c103680 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc371fde700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c10dda0 con 0x7fc36c100600 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc371fde700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c10e0a0 con 0x7fc36c104020 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 0x7fc36c101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 0x7fc36c101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55784/0 (socket says 172.21.15.92:55784) 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36affd700 1 -- 172.21.15.92:0/4254683017 learned_addr learned my addr 172.21.15.92:0/4254683017 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36b7fe700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c100600 0x7fc36c102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:39.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36bfff700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc36c104020 0x7fc36c101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36affd700 1 -- 172.21.15.92:0/4254683017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc36c104020 msgr2=0x7fc36c101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36affd700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc36c104020 0x7fc36c101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36affd700 1 -- 172.21.15.92:0/4254683017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c100600 msgr2=0x7fc36c102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36affd700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c100600 0x7fc36c102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.950+0000 7fc36affd700 1 -- 172.21.15.92:0/4254683017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c102420 con 0x7fc36c103680 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.951+0000 7fc36affd700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 0x7fc36c101630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc360009bc0 tx=0x7fc360004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.951+0000 7fc368ff9700 1 -- 172.21.15.92:0/4254683017 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360021070 con 0x7fc36c103680 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.951+0000 7fc368ff9700 1 -- 172.21.15.92:0/4254683017 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc360007b80 con 0x7fc36c103680 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.951+0000 7fc368ff9700 1 -- 172.21.15.92:0/4254683017 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36001d450 con 0x7fc36c103680 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.951+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c1a2670 con 0x7fc36c103680 2024-03-19T01:52:39.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.951+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c1a2ad0 con 0x7fc36c103680 2024-03-19T01:52:39.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.952+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc36c068fd0 con 0x7fc36c103680 2024-03-19T01:52:39.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.954+0000 7fc368ff9700 1 -- 172.21.15.92:0/4254683017 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc360004000 con 0x7fc36c103680 2024-03-19T01:52:39.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.954+0000 7fc368ff9700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc3540a2890 0x7fc3540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:39.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.955+0000 7fc36b7fe700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc3540a2890 0x7fc3540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:39.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.955+0000 7fc368ff9700 1 -- 172.21.15.92:0/4254683017 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc36000f020 con 0x7fc36c103680 2024-03-19T01:52:39.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.955+0000 7fc36b7fe700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc3540a2890 0x7fc3540a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc36c069d20 tx=0x7fc364007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:39.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:39.958+0000 7fc368ff9700 1 -- 172.21.15.92:0/4254683017 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc36002a020 con 0x7fc36c103680 2024-03-19T01:52:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.227+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc36c02dc50 con 0x7fc36c103680 2024-03-19T01:52:40.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.227+0000 7fc368ff9700 1 -- 172.21.15.92:0/4254683017 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3090 (secure 0 0 0) 0x7fc36c02dc50 con 0x7fc36c103680 2024-03-19T01:52:40.228 INFO:teuthology.orchestra.run.smithi092.stdout:e26 2024-03-19T01:52:40.228 INFO:teuthology.orchestra.run.smithi092.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-19T01:52:40.228 INFO:teuthology.orchestra.run.smithi092.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T01:52:40.228 INFO:teuthology.orchestra.run.smithi092.stdout:legacy client fscid: 1 2024-03-19T01:52:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:Filesystem 'cephfs' (1) 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:fs_name cephfs 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 26 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-03-19T01:50:49.881618+0000 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-03-19T01:50:56.972139+0000 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:tableserver 0 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:root 0 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:session_timeout 300 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:session_autoclose 300 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:max_file_size 1099511627776 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:max_xattr_size 65536 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:required_client_features {} 2024-03-19T01:52:40.229 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure 0 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:last_failure_osd_epoch 0 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:max_mds 5 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:in 0,1,2,3,4 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:up {0=24493,1=24476,2=24475,3=24457,4=14601} 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:failed 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:damaged 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:stopped 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:data_pools [3] 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:metadata_pool 2 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:inline_data disabled 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:balancer 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:bal_rank_mask -1 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:standby_count_wanted 0 2024-03-19T01:52:40.230 INFO:teuthology.orchestra.run.smithi092.stdout:qdb_cluster leader: 24493 members: 24476,24493,24475,24457,14601 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.i{0:24493} state up:active seq 5 addr [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.h{0:24434} state up:standby-replay seq 1 addr [v2:172.21.15.144:6836/1367568659,v1:172.21.15.144:6837/1367568659] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b{1:24476} state up:active seq 4 addr [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.l{1:24415} state up:standby-replay seq 1 addr [v2:172.21.15.159:6832/3252974913,v1:172.21.15.159:6833/3252974913] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c{2:24475} state up:active seq 8 addr [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.e{2:24413} state up:standby-replay seq 1 addr [v2:172.21.15.144:6834/2148025721,v1:172.21.15.144:6835/2148025721] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.f{3:24457} state up:active seq 9 addr [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.k{3:24389} state up:standby-replay seq 1 addr [v2:172.21.15.144:6832/3429474924,v1:172.21.15.144:6833/3429474924] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a{4:14601} state up:active seq 5 addr [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.g{4:14589} state up:standby-replay seq 1 addr [v2:172.21.15.92:6838/1427243227,v1:172.21.15.92:6839/1427243227] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout:Standby daemons: 2024-03-19T01:52:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:40.232 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.j{-1:14517} state up:standby seq 1 addr [v2:172.21.15.92:6834/383124452,v1:172.21.15.92:6835/383124452] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.232 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d{-1:14529} state up:standby seq 1 addr [v2:172.21.15.92:6836/1871566039,v1:172.21.15.92:6837/1871566039] compat {c=[1],r=[1],i=[1fff]}] 2024-03-19T01:52:40.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.230+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc3540a2890 msgr2=0x7fc3540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.230+0000 7fc371fde700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc3540a2890 0x7fc3540a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc36c069d20 tx=0x7fc364007400 comp rx=0 tx=0).stop 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.230+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 msgr2=0x7fc36c101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.230+0000 7fc371fde700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 0x7fc36c101630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc360009bc0 tx=0x7fc360004d70 comp rx=0 tx=0).stop 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.231+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 shutdown_connections 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.231+0000 7fc371fde700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc3540a2890 0x7fc3540a4d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.231+0000 7fc371fde700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc36c104020 0x7fc36c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.231+0000 7fc371fde700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c100600 0x7fc36c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.231+0000 7fc371fde700 1 --2- 172.21.15.92:0/4254683017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc36c103680 0x7fc36c101630 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.231+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 >> 172.21.15.92:0/4254683017 conn(0x7fc36c0fbf80 msgr2=0x7fc36c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.231+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 shutdown_connections 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:40.231+0000 7fc371fde700 1 -- 172.21.15.92:0/4254683017 wait complete. 2024-03-19T01:52:40.233 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2024-03-19T01:52:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:40 smithi092 ceph-mon[62574]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:40.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:40 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4254683017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-19T01:52:40.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:40 smithi144 ceph-mon[67199]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:40.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:40 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4254683017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-19T01:52:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:40 smithi159 ceph-mon[67110]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:40 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4254683017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-19T01:52:41.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-19T01:52:41.303 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:42.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:42 smithi144 ceph-mon[67199]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:42.871 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:42 smithi092 ceph-mon[62574]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:42 smithi159 ceph-mon[67110]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:43.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.116+0000 7f4b3d915700 1 -- 172.21.15.92:0/4290967239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38108030 msgr2=0x7f4b3810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:43.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.116+0000 7f4b3d915700 1 --2- 172.21.15.92:0/4290967239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38108030 0x7f4b3810a4a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000b0a0 tx=0x7f4b3000b3b0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.117+0000 7f4b3d915700 1 -- 172.21.15.92:0/4290967239 shutdown_connections 2024-03-19T01:52:43.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.117+0000 7f4b3d915700 1 --2- 172.21.15.92:0/4290967239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38108030 0x7f4b3810a4a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.117+0000 7f4b3d915700 1 --2- 172.21.15.92:0/4290967239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b38102dd0 0x7f4b381051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.117+0000 7f4b3d915700 1 --2- 172.21.15.92:0/4290967239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b38105700 0x7f4b38107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.117+0000 7f4b3d915700 1 -- 172.21.15.92:0/4290967239 >> 172.21.15.92:0/4290967239 conn(0x7f4b380fc7e0 msgr2=0x7f4b380fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:43.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.117+0000 7f4b3d915700 1 -- 172.21.15.92:0/4290967239 shutdown_connections 2024-03-19T01:52:43.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.117+0000 7f4b3d915700 1 -- 172.21.15.92:0/4290967239 wait complete. 2024-03-19T01:52:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.117+0000 7f4b3d915700 1 Processor -- start 2024-03-19T01:52:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b3d915700 1 -- start start 2024-03-19T01:52:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b3d915700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38102dd0 0x7f4b38198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b3d915700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b38105700 0x7f4b38198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b3d915700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b38108030 0x7f4b3819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b3d915700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3810df50 con 0x7f4b38102dd0 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b3d915700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3810ddd0 con 0x7f4b38108030 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b3d915700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3810e0d0 con 0x7f4b38105700 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b36ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38102dd0 0x7f4b38198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b36ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38102dd0 0x7f4b38198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55798/0 (socket says 172.21.15.92:55798) 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b36ffd700 1 -- 172.21.15.92:0/280259201 learned_addr learned my addr 172.21.15.92:0/280259201 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b367fc700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b38105700 0x7f4b38198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b377fe700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b38108030 0x7f4b3819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b36ffd700 1 -- 172.21.15.92:0/280259201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b38105700 msgr2=0x7f4b38198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b36ffd700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b38105700 0x7f4b38198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b36ffd700 1 -- 172.21.15.92:0/280259201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b38108030 msgr2=0x7f4b3819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b36ffd700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b38108030 0x7f4b3819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.118+0000 7f4b36ffd700 1 -- 172.21.15.92:0/280259201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b381a01c0 con 0x7f4b38102dd0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.119+0000 7f4b36ffd700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38102dd0 0x7f4b38198560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e9f0 tx=0x7f4b2800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.119+0000 7f4b3c913700 1 -- 172.21.15.92:0/280259201 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800cbd0 con 0x7f4b38102dd0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.119+0000 7f4b3c913700 1 -- 172.21.15.92:0/280259201 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2800ee90 con 0x7f4b38102dd0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.119+0000 7f4b3c913700 1 -- 172.21.15.92:0/280259201 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280056c0 con 0x7f4b38102dd0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.119+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b381a04b0 con 0x7f4b38102dd0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.119+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b381a09a0 con 0x7f4b38102dd0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.120+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b38068fd0 con 0x7f4b38102dd0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.121+0000 7f4b3c913700 1 -- 172.21.15.92:0/280259201 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4b28004750 con 0x7f4b38102dd0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.121+0000 7f4b3c913700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4b200a2890 0x7f4b200a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.122+0000 7f4b3c913700 1 -- 172.21.15.92:0/280259201 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4b280cef00 con 0x7f4b38102dd0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.122+0000 7f4b367fc700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4b200a2890 0x7f4b200a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:43.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.122+0000 7f4b367fc700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4b200a2890 0x7f4b200a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c007be0 tx=0x7f4b2c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:43.125 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.125+0000 7f4b3c913700 1 -- 172.21.15.92:0/280259201 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4b28053740 con 0x7f4b38102dd0 2024-03-19T01:52:43.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.325+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f4b3802dc50 con 0x7f4b38102dd0 2024-03-19T01:52:43.326 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.326+0000 7f4b3c913700 1 -- 172.21.15.92:0/280259201 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5297 (secure 0 0 0) 0x7f4b2809bfb0 con 0x7f4b38102dd0 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 87 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:fsid d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-03-19T01:42:44.963218+0000 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-03-19T01:51:40.581546+0000 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:crush_version 26 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:full_ratio 0.95 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:backfillfull_ratio 0.9 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:nearfull_ratio 0.85 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:require_min_compat_client luminous 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:min_compat_client luminous 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:require_osd_release squid 2024-03-19T01:52:43.327 INFO:teuthology.orchestra.run.smithi092.stdout:stretch_mode_enabled false 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:max_osd 12 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6802/2208105319,v1:172.21.15.92:6803/2208105319] [v2:172.21.15.92:6804/2208105319,v1:172.21.15.92:6805/2208105319] exists,up 42923908-6c69-4faf-8e1d-36ad4aa578dc 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/3690180636,v1:172.21.15.144:6801/3690180636] [v2:172.21.15.144:6802/3690180636,v1:172.21.15.144:6803/3690180636] exists,up 2481d494-0d38-4da0-a314-002c20836f0c 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/2365994247,v1:172.21.15.159:6801/2365994247] [v2:172.21.15.159:6802/2365994247,v1:172.21.15.159:6803/2365994247] exists,up 4e6cfae2-9e8c-4588-aa01-0fc24c7297d3 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6810/1878896189,v1:172.21.15.92:6811/1878896189] [v2:172.21.15.92:6812/1878896189,v1:172.21.15.92:6813/1878896189] exists,up eacfff22-7837-4e31-a0da-5d487d903923 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/3335611913,v1:172.21.15.144:6809/3335611913] [v2:172.21.15.144:6810/3335611913,v1:172.21.15.144:6811/3335611913] exists,up a1f152f6-a121-47d6-8169-0381921ac5ee 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/3359154626,v1:172.21.15.159:6809/3359154626] [v2:172.21.15.159:6810/3359154626,v1:172.21.15.159:6811/3359154626] exists,up 5e86f624-5f0c-4fc4-a12d-1d47750a4428 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6818/1165350802,v1:172.21.15.92:6819/1165350802] [v2:172.21.15.92:6820/1165350802,v1:172.21.15.92:6821/1165350802] exists,up e8595632-012c-480c-9324-994266f9a4d2 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6816/2720356070,v1:172.21.15.144:6817/2720356070] [v2:172.21.15.144:6818/2720356070,v1:172.21.15.144:6819/2720356070] exists,up 64c441ee-0a42-4022-bb95-ceef4053acac 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/3217714611,v1:172.21.15.159:6817/3217714611] [v2:172.21.15.159:6818/3217714611,v1:172.21.15.159:6819/3217714611] exists,up 73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6826/4029500126,v1:172.21.15.92:6827/4029500126] [v2:172.21.15.92:6828/4029500126,v1:172.21.15.92:6829/4029500126] exists,up 32cfda0d-94d8-4310-be83-3c0348ef3486 2024-03-19T01:52:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6824/3069035738,v1:172.21.15.144:6825/3069035738] [v2:172.21.15.144:6826/3069035738,v1:172.21.15.144:6827/3069035738] exists,up 8e70e11e-c96c-48bb-916a-4ca295db76e1 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/3616019688,v1:172.21.15.159:6825/3616019688] [v2:172.21.15.159:6826/3616019688,v1:172.21.15.159:6827/3616019688] exists,up 42fbc0b6-7285-451d-8c11-cb4179823811 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 2.13 [2,11] 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 2.1c [8,11] 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.c [3,9] 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.e [4,7] 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.17 [4,6] 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.20 [5,9] 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.31 [4,8] 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:pg_upmap_items 3.38 [3,9] 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/1888848283 expires 2024-03-20T01:43:39.363114+0000 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6801/1866676846 expires 2024-03-20T01:43:39.363114+0000 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6800/1866676846 expires 2024-03-20T01:43:39.363114+0000 2024-03-19T01:52:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6800/1465441882 expires 2024-03-20T01:43:12.221209+0000 2024-03-19T01:52:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/239005224 expires 2024-03-20T01:43:12.221209+0000 2024-03-19T01:52:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/2465427523 expires 2024-03-20T01:43:39.363114+0000 2024-03-19T01:52:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:6801/1465441882 expires 2024-03-20T01:43:12.221209+0000 2024-03-19T01:52:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/3016291750 expires 2024-03-20T01:43:12.221209+0000 2024-03-19T01:52:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/3316636845 expires 2024-03-20T01:43:39.363114+0000 2024-03-19T01:52:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:blocklist 172.21.15.92:0/3619099089 expires 2024-03-20T01:43:12.221209+0000 2024-03-19T01:52:43.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4b200a2890 msgr2=0x7f4b200a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:43.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4b200a2890 0x7f4b200a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c007be0 tx=0x7f4b2c0073d0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.331 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38102dd0 msgr2=0x7f4b38198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:43.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38102dd0 0x7f4b38198560 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e9f0 tx=0x7f4b2800ed00 comp rx=0 tx=0).stop 2024-03-19T01:52:43.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 shutdown_connections 2024-03-19T01:52:43.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4b200a2890 0x7f4b200a4d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b38108030 0x7f4b3819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b38102dd0 0x7f4b38198560 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 --2- 172.21.15.92:0/280259201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b38105700 0x7f4b38198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:43.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.329+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 >> 172.21.15.92:0/280259201 conn(0x7f4b380fc7e0 msgr2=0x7f4b38103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:43.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.330+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 shutdown_connections 2024-03-19T01:52:43.332 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:43.330+0000 7f4b3d915700 1 -- 172.21.15.92:0/280259201 wait complete. 2024-03-19T01:52:43.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:43 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/280259201' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-19T01:52:43.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:43 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/280259201' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-19T01:52:44.022 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:43 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/280259201' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-19T01:52:44.050 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-19T01:52:44.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:52:44.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.167+0000 7f6379cff640 1 -- 172.21.15.92:0/3298257841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63741075a0 msgr2=0x7f6374109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.167+0000 7f6379cff640 1 --2- 172.21.15.92:0/3298257841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63741075a0 0x7f6374109990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f636c009ca0 tx=0x7f636c02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.167+0000 7f6379cff640 1 -- 172.21.15.92:0/3298257841 shutdown_connections 2024-03-19T01:52:44.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.167+0000 7f6379cff640 1 --2- 172.21.15.92:0/3298257841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63741075a0 0x7f6374109990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.167+0000 7f6379cff640 1 --2- 172.21.15.92:0/3298257841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6374104c70 0x7f6374107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.167+0000 7f6379cff640 1 --2- 172.21.15.92:0/3298257841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6374102340 0x7f6374104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.167+0000 7f6379cff640 1 -- 172.21.15.92:0/3298257841 >> 172.21.15.92:0/3298257841 conn(0x7f63740fbcd0 msgr2=0x7f63740fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.167+0000 7f6379cff640 1 -- 172.21.15.92:0/3298257841 shutdown_connections 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.167+0000 7f6379cff640 1 -- 172.21.15.92:0/3298257841 wait complete. 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6379cff640 1 Processor -- start 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6379cff640 1 -- start start 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6379cff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6374102340 0x7f6374197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6379cff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6374104c70 0x7f6374198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6379cff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63741075a0 0x7f637419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6379cff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f637410d500 con 0x7f6374104c70 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6379cff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f637410d380 con 0x7f63741075a0 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6379cff640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f637410d680 con 0x7f6374102340 2024-03-19T01:52:44.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6374104c70 0x7f6374198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6374104c70 0x7f6374198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52792/0 (socket says 172.21.15.92:52792) 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6372ffd640 1 -- 172.21.15.92:0/2356794030 learned_addr learned my addr 172.21.15.92:0/2356794030 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f6373fff640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63741075a0 0x7f637419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.168+0000 7f63737fe640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6374102340 0x7f6374197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6372ffd640 1 -- 172.21.15.92:0/2356794030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6374102340 msgr2=0x7f6374197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6372ffd640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6374102340 0x7f6374197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6372ffd640 1 -- 172.21.15.92:0/2356794030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63741075a0 msgr2=0x7f637419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6372ffd640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63741075a0 0x7f637419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6372ffd640 1 -- 172.21.15.92:0/2356794030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637419f770 con 0x7f6374104c70 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6372ffd640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6374104c70 0x7f6374198090 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f636800e9f0 tx=0x7f636800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6370ff9640 1 -- 172.21.15.92:0/2356794030 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636800ccc0 con 0x7f6374104c70 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637419fa60 con 0x7f6374104c70 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6370ff9640 1 -- 172.21.15.92:0/2356794030 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6368005500 con 0x7f6374104c70 2024-03-19T01:52:44.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6370ff9640 1 -- 172.21.15.92:0/2356794030 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368010690 con 0x7f6374104c70 2024-03-19T01:52:44.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.169+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637419ffb0 con 0x7f6374104c70 2024-03-19T01:52:44.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.170+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63740fd7c0 con 0x7f6374104c70 2024-03-19T01:52:44.173 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.171+0000 7f6370ff9640 1 -- 172.21.15.92:0/2356794030 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f63680046c0 con 0x7f6374104c70 2024-03-19T01:52:44.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.172+0000 7f6370ff9640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f635c0a2800 0x7f635c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.172+0000 7f63737fe640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f635c0a2800 0x7f635c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.172+0000 7f6370ff9640 1 -- 172.21.15.92:0/2356794030 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f63680ced90 con 0x7f6374104c70 2024-03-19T01:52:44.174 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.172+0000 7f63737fe640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f635c0a2800 0x7f635c0a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6364009bd0 tx=0x7f6364005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:44.176 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.176+0000 7f6370ff9640 1 -- 172.21.15.92:0/2356794030 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f636809be40 con 0x7f6374104c70 2024-03-19T01:52:44.361 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.361+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6374062420 con 0x7f6374104c70 2024-03-19T01:52:44.362 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.362+0000 7f6370ff9640 1 -- 172.21.15.92:0/2356794030 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22690 (secure 0 0 0) 0x7f636809b740 con 0x7f6374104c70 2024-03-19T01:52:44.363 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:44.363 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":87,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:51:40.581546+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:52:44.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f635c0a2800 msgr2=0x7f635c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f635c0a2800 0x7f635c0a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6364009bd0 tx=0x7f6364005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:44.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6374104c70 msgr2=0x7f6374198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6374104c70 0x7f6374198090 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f636800e9f0 tx=0x7f636800eeb0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 shutdown_connections 2024-03-19T01:52:44.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63741075a0 0x7f637419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6374104c70 0x7f6374198090 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f635c0a2800 0x7f635c0a4cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 --2- 172.21.15.92:0/2356794030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6374102340 0x7f6374197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.364+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 >> 172.21.15.92:0/2356794030 conn(0x7f63740fbcd0 msgr2=0x7f6374102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:44.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.365+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 shutdown_connections 2024-03-19T01:52:44.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.365+0000 7f6379cff640 1 -- 172.21.15.92:0/2356794030 wait complete. 2024-03-19T01:52:44.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:46:07.695753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:50:47.888441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:50:49.068308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:52:44.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.482+0000 7f9daa873640 1 -- 172.21.15.92:0/3572413719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4101910 msgr2=0x7f9da4101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.482+0000 7f9daa873640 1 --2- 172.21.15.92:0/3572413719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4101910 0x7f9da4101d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009860 tx=0x7f9d9802bb70 comp rx=0 tx=0).stop 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.482+0000 7f9daa873640 1 -- 172.21.15.92:0/3572413719 shutdown_connections 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.482+0000 7f9daa873640 1 --2- 172.21.15.92:0/3572413719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da41022d0 0x7f9da4108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.482+0000 7f9daa873640 1 --2- 172.21.15.92:0/3572413719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4101910 0x7f9da4101d90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.482+0000 7f9daa873640 1 --2- 172.21.15.92:0/3572413719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9da4100720 0x7f9da4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.482+0000 7f9daa873640 1 -- 172.21.15.92:0/3572413719 >> 172.21.15.92:0/3572413719 conn(0x7f9da40fbcb0 msgr2=0x7f9da40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.482+0000 7f9daa873640 1 -- 172.21.15.92:0/3572413719 shutdown_connections 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.482+0000 7f9daa873640 1 -- 172.21.15.92:0/3572413719 wait complete. 2024-03-19T01:52:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9daa873640 1 Processor -- start 2024-03-19T01:52:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9daa873640 1 -- start start 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9daa873640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4100720 0x7f9da4197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9daa873640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4101910 0x7f9da41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9daa873640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9da41022d0 0x7f9da419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9daa873640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9da410c3a0 con 0x7f9da41022d0 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9daa873640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9da410c220 con 0x7f9da4100720 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9daa873640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9da410c520 con 0x7f9da4101910 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9da8e10640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9da41022d0 0x7f9da419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9da8e10640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9da41022d0 0x7f9da419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52808/0 (socket says 172.21.15.92:52808) 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9da8e10640 1 -- 172.21.15.92:0/4063330974 learned_addr learned my addr 172.21.15.92:0/4063330974 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9da37fe640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4101910 0x7f9da41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.483+0000 7f9da3fff640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4100720 0x7f9da4197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9da8e10640 1 -- 172.21.15.92:0/4063330974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4101910 msgr2=0x7f9da41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9da8e10640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4101910 0x7f9da41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9da8e10640 1 -- 172.21.15.92:0/4063330974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4100720 msgr2=0x7f9da4197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9da8e10640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4100720 0x7f9da4197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9da8e10640 1 -- 172.21.15.92:0/4063330974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da419f7c0 con 0x7f9da41022d0 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9da8e10640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9da41022d0 0x7f9da419f150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00eb10 tx=0x7f9d9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9da17fa640 1 -- 172.21.15.92:0/4063330974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c004b20 con 0x7f9da41022d0 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da419fab0 con 0x7f9da41022d0 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9da17fa640 1 -- 172.21.15.92:0/4063330974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c004cc0 con 0x7f9da41022d0 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9da17fa640 1 -- 172.21.15.92:0/4063330974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c010760 con 0x7f9da41022d0 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.484+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da41a0020 con 0x7f9da41022d0 2024-03-19T01:52:44.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.485+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da41a02b0 con 0x7f9da41022d0 2024-03-19T01:52:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.487+0000 7f9da17fa640 1 -- 172.21.15.92:0/4063330974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9d9c005020 con 0x7f9da41022d0 2024-03-19T01:52:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.487+0000 7f9da17fa640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d8c0a2800 0x7f9d8c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.487+0000 7f9da17fa640 1 -- 172.21.15.92:0/4063330974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9d9c0cf160 con 0x7f9da41022d0 2024-03-19T01:52:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.487+0000 7f9da3fff640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d8c0a2800 0x7f9d8c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.488+0000 7f9da3fff640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d8c0a2800 0x7f9d8c0a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009530 tx=0x7f9d94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:44.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.492+0000 7f9da17fa640 1 -- 172.21.15.92:0/4063330974 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9d9c09c210 con 0x7f9da41022d0 2024-03-19T01:52:44.689 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.689+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9da41a0480 con 0x7f9da41022d0 2024-03-19T01:52:44.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.689+0000 7f9da17fa640 1 -- 172.21.15.92:0/4063330974 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f9d9c09bb10 con 0x7f9da41022d0 2024-03-19T01:52:44.690 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-03-19T01:52:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.691+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d8c0a2800 msgr2=0x7f9d8c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.691+0000 7f9daa873640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d8c0a2800 0x7f9d8c0a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009530 tx=0x7f9d94009260 comp rx=0 tx=0).stop 2024-03-19T01:52:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.691+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9da41022d0 msgr2=0x7f9da419f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.691+0000 7f9daa873640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9da41022d0 0x7f9da419f150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00eb10 tx=0x7f9d9c00c2d0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.692+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 shutdown_connections 2024-03-19T01:52:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.692+0000 7f9daa873640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9da41022d0 0x7f9da419f150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.692+0000 7f9daa873640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f9d8c0a2800 0x7f9d8c0a4cc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.692+0000 7f9daa873640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4101910 0x7f9da41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.692+0000 7f9daa873640 1 --2- 172.21.15.92:0/4063330974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da4100720 0x7f9da4197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.692+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 >> 172.21.15.92:0/4063330974 conn(0x7f9da40fbcb0 msgr2=0x7f9da4068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.692+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 shutdown_connections 2024-03-19T01:52:44.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.692+0000 7f9daa873640 1 -- 172.21.15.92:0/4063330974 wait complete. 2024-03-19T01:52:44.702 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:44 smithi092 ceph-mon[62574]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:44.702 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:44 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2356794030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:44.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.808+0000 7f8826cd1640 1 -- 172.21.15.92:0/2874656020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 msgr2=0x7f8820100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.808+0000 7f8826cd1640 1 --2- 172.21.15.92:0/2874656020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 0x7f8820100b40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f881c009900 tx=0x7f881c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.808+0000 7f8826cd1640 1 -- 172.21.15.92:0/2874656020 shutdown_connections 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.808+0000 7f8826cd1640 1 --2- 172.21.15.92:0/2874656020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88201022d0 0x7f8820108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.808+0000 7f8826cd1640 1 --2- 172.21.15.92:0/2874656020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8820101910 0x7f8820101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.808+0000 7f8826cd1640 1 --2- 172.21.15.92:0/2874656020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 0x7f8820100b40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.808+0000 7f8826cd1640 1 -- 172.21.15.92:0/2874656020 >> 172.21.15.92:0/2874656020 conn(0x7f88200fbcb0 msgr2=0x7f88200fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.808+0000 7f8826cd1640 1 -- 172.21.15.92:0/2874656020 shutdown_connections 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.808+0000 7f8826cd1640 1 -- 172.21.15.92:0/2874656020 wait complete. 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8826cd1640 1 Processor -- start 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8826cd1640 1 -- start start 2024-03-19T01:52:44.811 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8826cd1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 0x7f88200694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8826cd1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8820101910 0x7f882019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8826cd1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88201022d0 0x7f882019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8826cd1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f882010c3a0 con 0x7f8820100720 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8826cd1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f882010c220 con 0x7f8820101910 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8826cd1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f882010c520 con 0x7f88201022d0 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8824a6d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 0x7f88200694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8824a6d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 0x7f88200694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52826/0 (socket says 172.21.15.92:52826) 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8824a6d640 1 -- 172.21.15.92:0/3465002170 learned_addr learned my addr 172.21.15.92:0/3465002170 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f882526e640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88201022d0 0x7f882019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8817fff640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8820101910 0x7f882019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8824a6d640 1 -- 172.21.15.92:0/3465002170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88201022d0 msgr2=0x7f882019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8824a6d640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88201022d0 0x7f882019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8824a6d640 1 -- 172.21.15.92:0/3465002170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8820101910 msgr2=0x7f882019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:44.812 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8824a6d640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8820101910 0x7f882019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:44.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.809+0000 7f8824a6d640 1 -- 172.21.15.92:0/3465002170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882019f7d0 con 0x7f8820100720 2024-03-19T01:52:44.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.810+0000 7f8824a6d640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 0x7f88200694f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f881c000c00 tx=0x7f881c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:44.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.810+0000 7f8815ffb640 1 -- 172.21.15.92:0/3465002170 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c041070 con 0x7f8820100720 2024-03-19T01:52:44.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.810+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882019fa00 con 0x7f8820100720 2024-03-19T01:52:44.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.810+0000 7f8815ffb640 1 -- 172.21.15.92:0/3465002170 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f881c004450 con 0x7f8820100720 2024-03-19T01:52:44.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.810+0000 7f8815ffb640 1 -- 172.21.15.92:0/3465002170 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c034440 con 0x7f8820100720 2024-03-19T01:52:44.813 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.810+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882019ff40 con 0x7f8820100720 2024-03-19T01:52:44.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.811+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8820107ab0 con 0x7f8820100720 2024-03-19T01:52:44.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.812+0000 7f8815ffb640 1 -- 172.21.15.92:0/3465002170 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f881c044070 con 0x7f8820100720 2024-03-19T01:52:44.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.813+0000 7f8815ffb640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f88080a2800 0x7f88080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:44.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.813+0000 7f8817fff640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f88080a2800 0x7f88080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:44.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.813+0000 7f8815ffb640 1 -- 172.21.15.92:0/3465002170 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f881c0f0500 con 0x7f8820100720 2024-03-19T01:52:44.814 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.813+0000 7f8817fff640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f88080a2800 0x7f88080a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8820069260 tx=0x7f8810005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:44.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:44.817+0000 7f8815ffb640 1 -- 172.21.15.92:0/3465002170 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f881c0bcff0 con 0x7f8820100720 2024-03-19T01:52:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:44 smithi144 ceph-mon[67199]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:44.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:44 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2356794030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:44 smithi159 ceph-mon[67110]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:44 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2356794030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:45.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.052+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88200624c0 con 0x7f8820100720 2024-03-19T01:52:45.053 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.053+0000 7f8815ffb640 1 -- 172.21.15.92:0/3465002170 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f881c0c1090 con 0x7f8820100720 2024-03-19T01:52:45.053 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-03-19T01:52:45.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f88080a2800 msgr2=0x7f88080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f88080a2800 0x7f88080a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8820069260 tx=0x7f8810005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:45.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 msgr2=0x7f88200694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 0x7f88200694f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f881c000c00 tx=0x7f881c02c640 comp rx=0 tx=0).stop 2024-03-19T01:52:45.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 shutdown_connections 2024-03-19T01:52:45.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f88080a2800 0x7f88080a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88201022d0 0x7f882019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8820101910 0x7f882019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 --2- 172.21.15.92:0/3465002170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8820100720 0x7f88200694f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 >> 172.21.15.92:0/3465002170 conn(0x7f88200fbcb0 msgr2=0x7f8820104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:45.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 shutdown_connections 2024-03-19T01:52:45.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.055+0000 7f8826cd1640 1 -- 172.21.15.92:0/3465002170 wait complete. 2024-03-19T01:52:45.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:52:45.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.183+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/975810222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74100720 msgr2=0x7f6c74100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.183+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/975810222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74100720 0x7f6c74100b40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009900 tx=0x7f6c6802bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:45.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.183+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/975810222 shutdown_connections 2024-03-19T01:52:45.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.183+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/975810222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c741022d0 0x7f6c74108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.184 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.183+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/975810222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c74101910 0x7f6c74101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.183+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/975810222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74100720 0x7f6c74100b40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.183+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/975810222 >> 172.21.15.92:0/975810222 conn(0x7f6c740fbcb0 msgr2=0x7f6c740fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:45.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.183+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/975810222 shutdown_connections 2024-03-19T01:52:45.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.183+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/975810222 wait complete. 2024-03-19T01:52:45.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c7a8bd640 1 Processor -- start 2024-03-19T01:52:45.185 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c7a8bd640 1 -- start start 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c7a8bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c74100720 0x7f6c74196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c7a8bd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c74101910 0x7f6c74197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c7a8bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c741022d0 0x7f6c7419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c7a8bd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410c390 con 0x7f6c741022d0 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c7a8bd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410c210 con 0x7f6c74100720 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c7a8bd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410c510 con 0x7f6c74101910 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c78e5a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c741022d0 0x7f6c7419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.184+0000 7f6c78e5a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c741022d0 0x7f6c7419e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52848/0 (socket says 172.21.15.92:52848) 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c78e5a640 1 -- 172.21.15.92:0/848404232 learned_addr learned my addr 172.21.15.92:0/848404232 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:45.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c737fe640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c74101910 0x7f6c74197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c73fff640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c74100720 0x7f6c74196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c78e5a640 1 -- 172.21.15.92:0/848404232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c74101910 msgr2=0x7f6c74197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c78e5a640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c74101910 0x7f6c74197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c78e5a640 1 -- 172.21.15.92:0/848404232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c74100720 msgr2=0x7f6c74196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c78e5a640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c74100720 0x7f6c74196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c78e5a640 1 -- 172.21.15.92:0/848404232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7419e6e0 con 0x7f6c741022d0 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c78e5a640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c741022d0 0x7f6c7419e070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00efd0 tx=0x7f6c6c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c717fa640 1 -- 172.21.15.92:0/848404232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c003b70 con 0x7f6c741022d0 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c717fa640 1 -- 172.21.15.92:0/848404232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c003d10 con 0x7f6c741022d0 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c717fa640 1 -- 172.21.15.92:0/848404232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0109f0 con 0x7f6c741022d0 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7419e9d0 con 0x7f6c741022d0 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.185+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7419eed0 con 0x7f6c741022d0 2024-03-19T01:52:45.188 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.186+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c74107ab0 con 0x7f6c741022d0 2024-03-19T01:52:45.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.187+0000 7f6c717fa640 1 -- 172.21.15.92:0/848404232 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f6c6c010b90 con 0x7f6c741022d0 2024-03-19T01:52:45.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.188+0000 7f6c717fa640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6c5c0a2800 0x7f6c5c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.188+0000 7f6c73fff640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6c5c0a2800 0x7f6c5c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.188+0000 7f6c717fa640 1 -- 172.21.15.92:0/848404232 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6c6c0cf250 con 0x7f6c741022d0 2024-03-19T01:52:45.189 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.188+0000 7f6c73fff640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6c5c0a2800 0x7f6c5c0a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6c68000c00 tx=0x7f6c68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:45.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.191+0000 7f6c717fa640 1 -- 172.21.15.92:0/848404232 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6c6c09c300 con 0x7f6c741022d0 2024-03-19T01:52:45.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.367+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6c7402d440 con 0x7f6c741022d0 2024-03-19T01:52:45.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.367+0000 7f6c717fa640 1 -- 172.21.15.92:0/848404232 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f6c6c09bc00 con 0x7f6c741022d0 2024-03-19T01:52:45.367 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-03-19T01:52:45.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6c5c0a2800 msgr2=0x7f6c5c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6c5c0a2800 0x7f6c5c0a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6c68000c00 tx=0x7f6c68005fd0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c741022d0 msgr2=0x7f6c7419e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c741022d0 0x7f6c7419e070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00efd0 tx=0x7f6c6c00c6a0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 shutdown_connections 2024-03-19T01:52:45.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c741022d0 0x7f6c7419e070 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6c5c0a2800 0x7f6c5c0a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c74101910 0x7f6c74197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 --2- 172.21.15.92:0/848404232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c74100720 0x7f6c74196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 >> 172.21.15.92:0/848404232 conn(0x7f6c740fbcb0 msgr2=0x7f6c74104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:45.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 shutdown_connections 2024-03-19T01:52:45.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.370+0000 7f6c7a8bd640 1 -- 172.21.15.92:0/848404232 wait complete. 2024-03-19T01:52:45.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:52:45.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.505+0000 7f3c1f036640 1 -- 172.21.15.92:0/3079205905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18100740 msgr2=0x7f3c18100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.505+0000 7f3c1f036640 1 --2- 172.21.15.92:0/3079205905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18100740 0x7f3c18100b60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009ca0 tx=0x7f3c1002c0f0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.508 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.505+0000 7f3c1f036640 1 -- 172.21.15.92:0/3079205905 shutdown_connections 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.505+0000 7f3c1f036640 1 --2- 172.21.15.92:0/3079205905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c181022f0 0x7f3c18108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.505+0000 7f3c1f036640 1 --2- 172.21.15.92:0/3079205905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c18101930 0x7f3c18101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.505+0000 7f3c1f036640 1 --2- 172.21.15.92:0/3079205905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18100740 0x7f3c18100b60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.505+0000 7f3c1f036640 1 -- 172.21.15.92:0/3079205905 >> 172.21.15.92:0/3079205905 conn(0x7f3c180fbcb0 msgr2=0x7f3c180fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.505+0000 7f3c1f036640 1 -- 172.21.15.92:0/3079205905 shutdown_connections 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.505+0000 7f3c1f036640 1 -- 172.21.15.92:0/3079205905 wait complete. 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1f036640 1 Processor -- start 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1f036640 1 -- start start 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1f036640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c18100740 0x7f3c180694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1f036640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18101930 0x7f3c1819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1f036640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c181022f0 0x7f3c1819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1f036640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1810c3b0 con 0x7f3c18101930 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1f036640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1810c230 con 0x7f3c181022f0 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1f036640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1810c530 con 0x7f3c18100740 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18101930 0x7f3c1819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1cdd2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c18100740 0x7f3c180694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18101930 0x7f3c1819c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52858/0 (socket says 172.21.15.92:52858) 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c17fff640 1 -- 172.21.15.92:0/501405140 learned_addr learned my addr 172.21.15.92:0/501405140 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1d5d3640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c181022f0 0x7f3c1819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c17fff640 1 -- 172.21.15.92:0/501405140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c18100740 msgr2=0x7f3c180694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c17fff640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c18100740 0x7f3c180694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.510 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c17fff640 1 -- 172.21.15.92:0/501405140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c181022f0 msgr2=0x7f3c1819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c17fff640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c181022f0 0x7f3c1819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c17fff640 1 -- 172.21.15.92:0/501405140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1819f7f0 con 0x7f3c18101930 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.506+0000 7f3c1cdd2640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c18100740 0x7f3c180694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.507+0000 7f3c17fff640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18101930 0x7f3c1819c8c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000e9f0 tx=0x7f3c0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.507+0000 7f3c15ffb640 1 -- 172.21.15.92:0/501405140 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000cd80 con 0x7f3c18101930 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.507+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1819fa80 con 0x7f3c18101930 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.507+0000 7f3c15ffb640 1 -- 172.21.15.92:0/501405140 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c00005500 con 0x7f3c18101930 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.507+0000 7f3c15ffb640 1 -- 172.21.15.92:0/501405140 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00010690 con 0x7f3c18101930 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.507+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1819ff80 con 0x7f3c18101930 2024-03-19T01:52:45.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.508+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c18107b30 con 0x7f3c18101930 2024-03-19T01:52:45.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.509+0000 7f3c15ffb640 1 -- 172.21.15.92:0/501405140 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f3c000046c0 con 0x7f3c18101930 2024-03-19T01:52:45.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.509+0000 7f3c15ffb640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3c040a2800 0x7f3c040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.509+0000 7f3c1cdd2640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3c040a2800 0x7f3c040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.509+0000 7f3c15ffb640 1 -- 172.21.15.92:0/501405140 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3c000cee40 con 0x7f3c18101930 2024-03-19T01:52:45.512 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.510+0000 7f3c1cdd2640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3c040a2800 0x7f3c040a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3c1002c700 tx=0x7f3c10032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:45.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.514+0000 7f3c15ffb640 1 -- 172.21.15.92:0/501405140 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3c0009bef0 con 0x7f3c18101930 2024-03-19T01:52:45.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:45 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4063330974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:45.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:45 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3465002170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:45.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:45 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/848404232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:45.816 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.816+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c18062420 con 0x7f3c18101930 2024-03-19T01:52:45.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.817+0000 7f3c15ffb640 1 -- 172.21.15.92:0/501405140 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11932 (secure 0 0 0) 0x7f3c0009b7f0 con 0x7f3c18101930 2024-03-19T01:52:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:45.818 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:50:49.881618+0000","modified":"2024-03-19T01:50:56.972139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24476,"mds_2":24475,"mds_3":24457,"mds_4":14601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6839/380126062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":380126062},{"type":"v1","addr":"172.21.15.144:6839","nonce":380126062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24476,24493,24475,24457,14601]},"id":1}]} 2024-03-19T01:52:45.818 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2024-03-19T01:52:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3c040a2800 msgr2=0x7f3c040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3c040a2800 0x7f3c040a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3c1002c700 tx=0x7f3c10032590 comp rx=0 tx=0).stop 2024-03-19T01:52:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18101930 msgr2=0x7f3c1819c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18101930 0x7f3c1819c8c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000e9f0 tx=0x7f3c0000edf0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 shutdown_connections 2024-03-19T01:52:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c181022f0 0x7f3c1819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c18101930 0x7f3c1819c8c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f3c040a2800 0x7f3c040a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 --2- 172.21.15.92:0/501405140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c18100740 0x7f3c180694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.819+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 >> 172.21.15.92:0/501405140 conn(0x7f3c180fbcb0 msgr2=0x7f3c18104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.820+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 shutdown_connections 2024-03-19T01:52:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.820+0000 7f3c1f036640 1 -- 172.21.15.92:0/501405140 wait complete. 2024-03-19T01:52:45.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:52:45.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:45 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4063330974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:45.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:45 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3465002170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:45.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:45 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/848404232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:45.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.933+0000 7ff469644640 1 -- 172.21.15.92:0/2239591764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 msgr2=0x7ff464108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.933+0000 7ff469644640 1 --2- 172.21.15.92:0/2239591764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 0x7ff464108b80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009ca0 tx=0x7ff45c02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.933+0000 7ff469644640 1 -- 172.21.15.92:0/2239591764 shutdown_connections 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.933+0000 7ff469644640 1 --2- 172.21.15.92:0/2239591764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 0x7ff464108b80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.933+0000 7ff469644640 1 --2- 172.21.15.92:0/2239591764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff464101930 0x7ff464101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.933+0000 7ff469644640 1 --2- 172.21.15.92:0/2239591764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff464100740 0x7ff464100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.933+0000 7ff469644640 1 -- 172.21.15.92:0/2239591764 >> 172.21.15.92:0/2239591764 conn(0x7ff4640fbcb0 msgr2=0x7ff4640fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.933+0000 7ff469644640 1 -- 172.21.15.92:0/2239591764 shutdown_connections 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.933+0000 7ff469644640 1 -- 172.21.15.92:0/2239591764 wait complete. 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff469644640 1 Processor -- start 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff469644640 1 -- start start 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff469644640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff464100740 0x7ff4641969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff469644640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff464101930 0x7ff464196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff469644640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 0x7ff46419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff469644640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410c3a0 con 0x7ff4641022f0 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff469644640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410c220 con 0x7ff464101930 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff469644640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410c520 con 0x7ff464100740 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff4637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 0x7ff46419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff4637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 0x7ff46419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52878/0 (socket says 172.21.15.92:52878) 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff4637fe640 1 -- 172.21.15.92:0/1908964539 learned_addr learned my addr 172.21.15.92:0/1908964539 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff462ffd640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff464100740 0x7ff4641969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff45a7fc640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff464101930 0x7ff464196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.934+0000 7ff4637fe640 1 -- 172.21.15.92:0/1908964539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff464100740 msgr2=0x7ff4641969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff4637fe640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff464100740 0x7ff4641969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff4637fe640 1 -- 172.21.15.92:0/1908964539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff464101930 msgr2=0x7ff464196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff4637fe640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff464101930 0x7ff464196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:45.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff4637fe640 1 -- 172.21.15.92:0/1908964539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46419e600 con 0x7ff4641022f0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff45a7fc640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff464101930 0x7ff464196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff4637fe640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 0x7ff46419df90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff45c037040 tx=0x7ff45c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff460ff9640 1 -- 172.21.15.92:0/1908964539 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c041020 con 0x7ff4641022f0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff460ff9640 1 -- 172.21.15.92:0/1908964539 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c034cd0 con 0x7ff4641022f0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff464068980 con 0x7ff4641022f0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff460ff9640 1 -- 172.21.15.92:0/1908964539 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c039040 con 0x7ff4641022f0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.935+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff464068e70 con 0x7ff4641022f0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.936+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff464068610 con 0x7ff4641022f0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.937+0000 7ff460ff9640 1 -- 172.21.15.92:0/1908964539 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff45c044020 con 0x7ff4641022f0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.938+0000 7ff460ff9640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff4500a2800 0x7ff4500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.938+0000 7ff462ffd640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff4500a2800 0x7ff4500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.938+0000 7ff460ff9640 1 -- 172.21.15.92:0/1908964539 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff45c0f08f0 con 0x7ff4641022f0 2024-03-19T01:52:45.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.938+0000 7ff462ffd640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff4500a2800 0x7ff4500a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff464101790 tx=0x7ff454005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:45.943 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:45.942+0000 7ff460ff9640 1 -- 172.21.15.92:0/1908964539 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff45c04b020 con 0x7ff4641022f0 2024-03-19T01:52:45.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:45 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4063330974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:45.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:45 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3465002170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:45.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:45 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/848404232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:46.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.196+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff46419e890 con 0x7ff4641022f0 2024-03-19T01:52:46.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.198+0000 7ff460ff9640 1 -- 172.21.15.92:0/1908964539 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22690 (secure 0 0 0) 0x7ff45c0bda30 con 0x7ff4641022f0 2024-03-19T01:52:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:46.199 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":87,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:51:40.581546+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:52:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff4500a2800 msgr2=0x7ff4500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff4500a2800 0x7ff4500a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff464101790 tx=0x7ff454005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 msgr2=0x7ff46419df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 0x7ff46419df90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff45c037040 tx=0x7ff45c032f40 comp rx=0 tx=0).stop 2024-03-19T01:52:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 shutdown_connections 2024-03-19T01:52:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4641022f0 0x7ff46419df90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff464101930 0x7ff464196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff4500a2800 0x7ff4500a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 --2- 172.21.15.92:0/1908964539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff464100740 0x7ff4641969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.200+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 >> 172.21.15.92:0/1908964539 conn(0x7ff4640fbcb0 msgr2=0x7ff464104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.201+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 shutdown_connections 2024-03-19T01:52:46.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.201+0000 7ff469644640 1 -- 172.21.15.92:0/1908964539 wait complete. 2024-03-19T01:52:46.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:46:07.695753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:50:47.888441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:50:49.068308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T01:52:46.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.333+0000 7efd258b2640 1 -- 172.21.15.92:0/2911545021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd201022d0 msgr2=0x7efd20108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.333+0000 7efd258b2640 1 --2- 172.21.15.92:0/2911545021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd201022d0 0x7efd20108b60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd10009900 tx=0x7efd1002bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.333+0000 7efd258b2640 1 -- 172.21.15.92:0/2911545021 shutdown_connections 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.333+0000 7efd258b2640 1 --2- 172.21.15.92:0/2911545021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd201022d0 0x7efd20108b60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.333+0000 7efd258b2640 1 --2- 172.21.15.92:0/2911545021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd20101910 0x7efd20101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.333+0000 7efd258b2640 1 --2- 172.21.15.92:0/2911545021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd20100720 0x7efd20100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.333+0000 7efd258b2640 1 -- 172.21.15.92:0/2911545021 >> 172.21.15.92:0/2911545021 conn(0x7efd200fbcb0 msgr2=0x7efd200fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.333+0000 7efd258b2640 1 -- 172.21.15.92:0/2911545021 shutdown_connections 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.333+0000 7efd258b2640 1 -- 172.21.15.92:0/2911545021 wait complete. 2024-03-19T01:52:46.336 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd258b2640 1 Processor -- start 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd258b2640 1 -- start start 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd258b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd20100720 0x7efd200694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd258b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd20101910 0x7efd2019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd258b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd201022d0 0x7efd2019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd258b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010c3a0 con 0x7efd20101910 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd258b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010c220 con 0x7efd20100720 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd258b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010c520 con 0x7efd201022d0 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd1e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd20101910 0x7efd2019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd1e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd20101910 0x7efd2019c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52900/0 (socket says 172.21.15.92:52900) 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd1e7fc640 1 -- 172.21.15.92:0/239774813 learned_addr learned my addr 172.21.15.92:0/239774813 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd1f7fe640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd201022d0 0x7efd2019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.334+0000 7efd1effd640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd20100720 0x7efd200694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd1e7fc640 1 -- 172.21.15.92:0/239774813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd201022d0 msgr2=0x7efd2019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd1e7fc640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd201022d0 0x7efd2019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd1e7fc640 1 -- 172.21.15.92:0/239774813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd20100720 msgr2=0x7efd200694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd1e7fc640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd20100720 0x7efd200694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd1e7fc640 1 -- 172.21.15.92:0/239774813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2019f790 con 0x7efd20101910 2024-03-19T01:52:46.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd1e7fc640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd20101910 0x7efd2019c860 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efd0800eb90 tx=0x7efd0800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:46.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd248b0640 1 -- 172.21.15.92:0/239774813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08004d20 con 0x7efd20101910 2024-03-19T01:52:46.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2019fa20 con 0x7efd20101910 2024-03-19T01:52:46.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd248b0640 1 -- 172.21.15.92:0/239774813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0800ee90 con 0x7efd20101910 2024-03-19T01:52:46.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd248b0640 1 -- 172.21.15.92:0/239774813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08010830 con 0x7efd20101910 2024-03-19T01:52:46.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.335+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2019ff70 con 0x7efd20101910 2024-03-19T01:52:46.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.336+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd20107a40 con 0x7efd20101910 2024-03-19T01:52:46.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.337+0000 7efd248b0640 1 -- 172.21.15.92:0/239774813 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7efd08005020 con 0x7efd20101910 2024-03-19T01:52:46.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.338+0000 7efd248b0640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd0c0a2800 0x7efd0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.338+0000 7efd1effd640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd0c0a2800 0x7efd0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.338+0000 7efd248b0640 1 -- 172.21.15.92:0/239774813 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efd080cefe0 con 0x7efd20101910 2024-03-19T01:52:46.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.339+0000 7efd1effd640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd0c0a2800 0x7efd0c0a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd180095f0 tx=0x7efd18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:46.343 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.343+0000 7efd248b0640 1 -- 172.21.15.92:0/239774813 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efd0809c130 con 0x7efd20101910 2024-03-19T01:52:46.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.522+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd2002d380 con 0x7efd20101910 2024-03-19T01:52:46.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.522+0000 7efd248b0640 1 -- 172.21.15.92:0/239774813 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7efd0809ba30 con 0x7efd20101910 2024-03-19T01:52:46.523 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-03-19T01:52:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd0c0a2800 msgr2=0x7efd0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd0c0a2800 0x7efd0c0a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd180095f0 tx=0x7efd18005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd20101910 msgr2=0x7efd2019c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd20101910 0x7efd2019c860 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efd0800eb90 tx=0x7efd0800c6a0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 shutdown_connections 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efd0c0a2800 0x7efd0c0a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd201022d0 0x7efd2019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd20101910 0x7efd2019c860 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 --2- 172.21.15.92:0/239774813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd20100720 0x7efd200694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 >> 172.21.15.92:0/239774813 conn(0x7efd200fbcb0 msgr2=0x7efd20104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 shutdown_connections 2024-03-19T01:52:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.524+0000 7efd258b2640 1 -- 172.21.15.92:0/239774813 wait complete. 2024-03-19T01:52:46.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T01:52:46.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.645+0000 7fac29990640 1 -- 172.21.15.92:0/621300395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24100730 msgr2=0x7fac24100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.645+0000 7fac29990640 1 --2- 172.21.15.92:0/621300395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24100730 0x7fac24100b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009ca0 tx=0x7fac1c02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.645+0000 7fac29990640 1 -- 172.21.15.92:0/621300395 shutdown_connections 2024-03-19T01:52:46.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.645+0000 7fac29990640 1 --2- 172.21.15.92:0/621300395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac241022e0 0x7fac24108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.645+0000 7fac29990640 1 --2- 172.21.15.92:0/621300395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac24101920 0x7fac24101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.645+0000 7fac29990640 1 --2- 172.21.15.92:0/621300395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24100730 0x7fac24100b50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.645+0000 7fac29990640 1 -- 172.21.15.92:0/621300395 >> 172.21.15.92:0/621300395 conn(0x7fac240fbcc0 msgr2=0x7fac240fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:46.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.645+0000 7fac29990640 1 -- 172.21.15.92:0/621300395 shutdown_connections 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.645+0000 7fac29990640 1 -- 172.21.15.92:0/621300395 wait complete. 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac29990640 1 Processor -- start 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac29990640 1 -- start start 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac29990640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac24100730 0x7fac24197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac29990640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac24101920 0x7fac241980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac29990640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac241022e0 0x7fac2419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac29990640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac2410c3a0 con 0x7fac241022e0 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac29990640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fac2410c220 con 0x7fac24100730 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac29990640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac2410c520 con 0x7fac24101920 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac241022e0 0x7fac2419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac22ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac24100730 0x7fac24197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac237fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac241022e0 0x7fac2419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52920/0 (socket says 172.21.15.92:52920) 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac237fe640 1 -- 172.21.15.92:0/3934657918 learned_addr learned my addr 172.21.15.92:0/3934657918 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:46.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac227fc640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac24101920 0x7fac241980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac22ffd640 1 -- 172.21.15.92:0/3934657918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac24101920 msgr2=0x7fac241980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac22ffd640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac24101920 0x7fac241980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac22ffd640 1 -- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac241022e0 msgr2=0x7fac2419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac22ffd640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac241022e0 0x7fac2419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac22ffd640 1 -- 172.21.15.92:0/3934657918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2419f7a0 con 0x7fac24100730 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.646+0000 7fac237fe640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac241022e0 0x7fac2419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.647+0000 7fac22ffd640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac24100730 0x7fac24197b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c630 tx=0x7fac1c032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.647+0000 7fac2898e640 1 -- 172.21.15.92:0/3934657918 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c041020 con 0x7fac24100730 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.647+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2419fa30 con 0x7fac24100730 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.647+0000 7fac2898e640 1 -- 172.21.15.92:0/3934657918 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1c034cd0 con 0x7fac24100730 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.647+0000 7fac2898e640 1 -- 172.21.15.92:0/3934657918 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c039040 con 0x7fac24100730 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.647+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2419ff20 con 0x7fac24100730 2024-03-19T01:52:46.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.648+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac241a0150 con 0x7fac24100730 2024-03-19T01:52:46.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.649+0000 7fac2898e640 1 -- 172.21.15.92:0/3934657918 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fac1c044020 con 0x7fac24100730 2024-03-19T01:52:46.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.650+0000 7fac2898e640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fac100a2800 0x7fac100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.650+0000 7fac227fc640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fac100a2800 0x7fac100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.650+0000 7fac2898e640 1 -- 172.21.15.92:0/3934657918 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fac1c0f0850 con 0x7fac24100730 2024-03-19T01:52:46.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.650+0000 7fac227fc640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fac100a2800 0x7fac100a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fac24199090 tx=0x7fac0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:46.655 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.655+0000 7fac2898e640 1 -- 172.21.15.92:0/3934657918 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fac1c04b020 con 0x7fac24100730 2024-03-19T01:52:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:46 smithi092 ceph-mon[62574]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:46 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/501405140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:46 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1908964539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:46 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/239774813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.836+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fac241a0320 con 0x7fac24100730 2024-03-19T01:52:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.837+0000 7fac2898e640 1 -- 172.21.15.92:0/3934657918 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fac1c0bd990 con 0x7fac24100730 2024-03-19T01:52:46.837 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fac100a2800 msgr2=0x7fac100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fac100a2800 0x7fac100a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fac24199090 tx=0x7fac0c005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac24100730 msgr2=0x7fac24197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac24100730 0x7fac24197b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02c630 tx=0x7fac1c032f20 comp rx=0 tx=0).stop 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 shutdown_connections 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac241022e0 0x7fac2419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fac100a2800 0x7fac100a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac24101920 0x7fac241980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 --2- 172.21.15.92:0/3934657918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac24100730 0x7fac24197b80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 >> 172.21.15.92:0/3934657918 conn(0x7fac240fbcc0 msgr2=0x7fac240fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 shutdown_connections 2024-03-19T01:52:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.839+0000 7fac29990640 1 -- 172.21.15.92:0/3934657918 wait complete. 2024-03-19T01:52:46.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:46 smithi144 ceph-mon[67199]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:46.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:46 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/501405140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:46.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:46 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1908964539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:46.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:46 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/239774813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:46.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T01:52:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:46 smithi159 ceph-mon[67110]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:46 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/501405140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:46 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1908964539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:46 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/239774813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T01:52:46.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.955+0000 7f2979d10640 1 -- 172.21.15.92:0/1287661710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974102360 msgr2=0x7f2974104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.955+0000 7f2979d10640 1 --2- 172.21.15.92:0/1287661710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974102360 0x7f2974104750 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2968009900 tx=0x7f296802bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:46.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.955+0000 7f2979d10640 1 -- 172.21.15.92:0/1287661710 shutdown_connections 2024-03-19T01:52:46.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.955+0000 7f2979d10640 1 --2- 172.21.15.92:0/1287661710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29741075c0 0x7f29741099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.955+0000 7f2979d10640 1 --2- 172.21.15.92:0/1287661710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974104c90 0x7f2974107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.955+0000 7f2979d10640 1 --2- 172.21.15.92:0/1287661710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974102360 0x7f2974104750 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.955+0000 7f2979d10640 1 -- 172.21.15.92:0/1287661710 >> 172.21.15.92:0/1287661710 conn(0x7f29740fbcb0 msgr2=0x7f29740fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:46.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.955+0000 7f2979d10640 1 -- 172.21.15.92:0/1287661710 shutdown_connections 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.955+0000 7f2979d10640 1 -- 172.21.15.92:0/1287661710 wait complete. 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2979d10640 1 Processor -- start 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2979d10640 1 -- start start 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2979d10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974102360 0x7f2974197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2979d10640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974104c90 0x7f2974198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2979d10640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29741075c0 0x7f297419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2979d10640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f297410d4d0 con 0x7f2974104c90 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2979d10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f297410d350 con 0x7f2974102360 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2979d10640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f297410d650 con 0x7f29741075c0 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2972ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974104c90 0x7f2974198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2972ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974104c90 0x7f2974198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52938/0 (socket says 172.21.15.92:52938) 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2972ffd640 1 -- 172.21.15.92:0/4189952974 learned_addr learned my addr 172.21.15.92:0/4189952974 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f2973fff640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29741075c0 0x7f297419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.956+0000 7f29737fe640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974102360 0x7f2974197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2972ffd640 1 -- 172.21.15.92:0/4189952974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29741075c0 msgr2=0x7f297419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2972ffd640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29741075c0 0x7f297419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2972ffd640 1 -- 172.21.15.92:0/4189952974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974102360 msgr2=0x7f2974197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2972ffd640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974102360 0x7f2974197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2972ffd640 1 -- 172.21.15.92:0/4189952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297419f7f0 con 0x7f2974104c90 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2972ffd640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974104c90 0x7f2974198110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f296400e9c0 tx=0x7f296400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2970ff9640 1 -- 172.21.15.92:0/4189952974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296400cd80 con 0x7f2974104c90 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2970ff9640 1 -- 172.21.15.92:0/4189952974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2964005500 con 0x7f2974104c90 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297419fae0 con 0x7f2974104c90 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2970ff9640 1 -- 172.21.15.92:0/4189952974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964010690 con 0x7f2974104c90 2024-03-19T01:52:46.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.957+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741a0030 con 0x7f2974104c90 2024-03-19T01:52:46.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.958+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29740fd7a0 con 0x7f2974104c90 2024-03-19T01:52:46.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.959+0000 7f2970ff9640 1 -- 172.21.15.92:0/4189952974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f29640046c0 con 0x7f2974104c90 2024-03-19T01:52:46.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.960+0000 7f2970ff9640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f295c0a2800 0x7f295c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:46.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.960+0000 7f29737fe640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f295c0a2800 0x7f295c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:46.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.960+0000 7f2970ff9640 1 -- 172.21.15.92:0/4189952974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f29640ced80 con 0x7f2974104c90 2024-03-19T01:52:46.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.960+0000 7f29737fe640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f295c0a2800 0x7f295c0a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f296802c7b0 tx=0x7f2968005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:46.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:46.964+0000 7f2970ff9640 1 -- 172.21.15.92:0/4189952974 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f296409be30 con 0x7f2974104c90 2024-03-19T01:52:47.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.144+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2974062420 con 0x7f2974104c90 2024-03-19T01:52:47.144 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.144+0000 7f2970ff9640 1 -- 172.21.15.92:0/4189952974 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f296409b730 con 0x7f2974104c90 2024-03-19T01:52:47.145 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-03-19T01:52:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f295c0a2800 msgr2=0x7f295c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f295c0a2800 0x7f295c0a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f296802c7b0 tx=0x7f2968005fd0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974104c90 msgr2=0x7f2974198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974104c90 0x7f2974198110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f296400e9c0 tx=0x7f296400ee80 comp rx=0 tx=0).stop 2024-03-19T01:52:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 shutdown_connections 2024-03-19T01:52:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f295c0a2800 0x7f295c0a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29741075c0 0x7f297419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2974104c90 0x7f2974198110 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 --2- 172.21.15.92:0/4189952974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974102360 0x7f2974197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 >> 172.21.15.92:0/4189952974 conn(0x7f29740fbcb0 msgr2=0x7f2974102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 shutdown_connections 2024-03-19T01:52:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.146+0000 7f2979d10640 1 -- 172.21.15.92:0/4189952974 wait complete. 2024-03-19T01:52:47.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T01:52:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.260+0000 7f1811fa9640 1 -- 172.21.15.92:0/402320304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c101930 msgr2=0x7f180c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.260+0000 7f1811fa9640 1 --2- 172.21.15.92:0/402320304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c101930 0x7f180c101db0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009900 tx=0x7f17f402bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 -- 172.21.15.92:0/402320304 shutdown_connections 2024-03-19T01:52:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 --2- 172.21.15.92:0/402320304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f180c1022f0 0x7f180c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 --2- 172.21.15.92:0/402320304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c101930 0x7f180c101db0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 --2- 172.21.15.92:0/402320304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f180c100740 0x7f180c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 -- 172.21.15.92:0/402320304 >> 172.21.15.92:0/402320304 conn(0x7f180c0fbcb0 msgr2=0x7f180c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 -- 172.21.15.92:0/402320304 shutdown_connections 2024-03-19T01:52:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 -- 172.21.15.92:0/402320304 wait complete. 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 Processor -- start 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 -- start start 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f180c100740 0x7f180c196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f180c101930 0x7f180c196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1022f0 0x7f180c19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f180c10c3c0 con 0x7f180c1022f0 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f180c10c240 con 0x7f180c100740 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.261+0000 7f1811fa9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f180c10c540 con 0x7f180c101930 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1022f0 0x7f180c19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1022f0 0x7f180c19e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52960/0 (socket says 172.21.15.92:52960) 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f180c100740 0x7f180c196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180bfff640 1 -- 172.21.15.92:0/2376522501 learned_addr learned my addr 172.21.15.92:0/2376522501 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f1803fff640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f180c101930 0x7f180c196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180bfff640 1 -- 172.21.15.92:0/2376522501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f180c101930 msgr2=0x7f180c196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180bfff640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f180c101930 0x7f180c196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180bfff640 1 -- 172.21.15.92:0/2376522501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f180c100740 msgr2=0x7f180c196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180bfff640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f180c100740 0x7f180c196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180bfff640 1 -- 172.21.15.92:0/2376522501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c19e680 con 0x7f180c1022f0 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180b7fe640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f180c100740 0x7f180c196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f180bfff640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1022f0 0x7f180c19e010 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f180400eca0 tx=0x7f180400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f18097fa640 1 -- 172.21.15.92:0/2376522501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804004b20 con 0x7f180c1022f0 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f18097fa640 1 -- 172.21.15.92:0/2376522501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1804004cc0 con 0x7f180c1022f0 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c0689e0 con 0x7f180c1022f0 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f18097fa640 1 -- 172.21.15.92:0/2376522501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804017650 con 0x7f180c1022f0 2024-03-19T01:52:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.262+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c068eb0 con 0x7f180c1022f0 2024-03-19T01:52:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.263+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180c068610 con 0x7f180c1022f0 2024-03-19T01:52:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.264+0000 7f18097fa640 1 -- 172.21.15.92:0/2376522501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1804005020 con 0x7f180c1022f0 2024-03-19T01:52:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.265+0000 7f18097fa640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17f80a2800 0x7f17f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.265+0000 7f180b7fe640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17f80a2800 0x7f17f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.265+0000 7f18097fa640 1 -- 172.21.15.92:0/2376522501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f18040d04b0 con 0x7f180c1022f0 2024-03-19T01:52:47.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.265+0000 7f180b7fe640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17f80a2800 0x7f17f80a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f180c101790 tx=0x7f17fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.269+0000 7f18097fa640 1 -- 172.21.15.92:0/2376522501 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f180409cf10 con 0x7f180c1022f0 2024-03-19T01:52:47.492 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.492+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f180c19e910 con 0x7f180c1022f0 2024-03-19T01:52:47.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.493+0000 7f18097fa640 1 -- 172.21.15.92:0/2376522501 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11932 (secure 0 0 0) 0x7f18040a0fb0 con 0x7f180c1022f0 2024-03-19T01:52:47.493 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:47.493 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:50:49.881618+0000","modified":"2024-03-19T01:50:56.972139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24476,"mds_2":24475,"mds_3":24457,"mds_4":14601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6839/380126062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":380126062},{"type":"v1","addr":"172.21.15.144:6839","nonce":380126062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24476,24493,24475,24457,14601]},"id":1}]} 2024-03-19T01:52:47.494 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2024-03-19T01:52:47.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17f80a2800 msgr2=0x7f17f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17f80a2800 0x7f17f80a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f180c101790 tx=0x7f17fc005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:47.495 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1022f0 msgr2=0x7f180c19e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1022f0 0x7f180c19e010 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f180400eca0 tx=0x7f180400b690 comp rx=0 tx=0).stop 2024-03-19T01:52:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 shutdown_connections 2024-03-19T01:52:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c1022f0 0x7f180c19e010 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f17f80a2800 0x7f17f80a4cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f180c101930 0x7f180c196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 --2- 172.21.15.92:0/2376522501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f180c100740 0x7f180c196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 >> 172.21.15.92:0/2376522501 conn(0x7f180c0fbcb0 msgr2=0x7f180c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 shutdown_connections 2024-03-19T01:52:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.495+0000 7f1811fa9640 1 -- 172.21.15.92:0/2376522501 wait complete. 2024-03-19T01:52:47.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T01:52:47.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.610+0000 7fec1aff4640 1 -- 172.21.15.92:0/3081276932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 msgr2=0x7fec14100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.610+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3081276932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 0x7fec14100b60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fec08009900 tx=0x7fec0802bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.610+0000 7fec1aff4640 1 -- 172.21.15.92:0/3081276932 shutdown_connections 2024-03-19T01:52:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.610+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3081276932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec141022f0 0x7fec14108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.610+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3081276932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec14101930 0x7fec14101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.610+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3081276932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 0x7fec14100b60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.610+0000 7fec1aff4640 1 -- 172.21.15.92:0/3081276932 >> 172.21.15.92:0/3081276932 conn(0x7fec140fbcb0 msgr2=0x7fec140fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.610+0000 7fec1aff4640 1 -- 172.21.15.92:0/3081276932 shutdown_connections 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.610+0000 7fec1aff4640 1 -- 172.21.15.92:0/3081276932 wait complete. 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec1aff4640 1 Processor -- start 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec1aff4640 1 -- start start 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec1aff4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 0x7fec14196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec1aff4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec14101930 0x7fec14196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec1aff4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec141022f0 0x7fec1419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec1aff4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec1410c3b0 con 0x7fec14100740 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec1aff4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec1410c230 con 0x7fec14101930 2024-03-19T01:52:47.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec1aff4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fec1410c530 con 0x7fec141022f0 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec18d90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 0x7fec14196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec18d90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 0x7fec14196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52980/0 (socket says 172.21.15.92:52980) 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec18d90640 1 -- 172.21.15.92:0/3622259959 learned_addr learned my addr 172.21.15.92:0/3622259959 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec13fff640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec14101930 0x7fec14196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.611+0000 7fec19591640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec141022f0 0x7fec1419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec18d90640 1 -- 172.21.15.92:0/3622259959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec141022f0 msgr2=0x7fec1419e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec18d90640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec141022f0 0x7fec1419e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec18d90640 1 -- 172.21.15.92:0/3622259959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec14101930 msgr2=0x7fec14196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec18d90640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec14101930 0x7fec14196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec18d90640 1 -- 172.21.15.92:0/3622259959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1419e6d0 con 0x7fec14100740 2024-03-19T01:52:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec13fff640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec14101930 0x7fec14196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec18d90640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 0x7fec14196ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fec08000c00 tx=0x7fec0802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec11ffb640 1 -- 172.21.15.92:0/3622259959 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08041070 con 0x7fec14100740 2024-03-19T01:52:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec11ffb640 1 -- 172.21.15.92:0/3622259959 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec08004450 con 0x7fec14100740 2024-03-19T01:52:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1419e960 con 0x7fec14100740 2024-03-19T01:52:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec11ffb640 1 -- 172.21.15.92:0/3622259959 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08034440 con 0x7fec14100740 2024-03-19T01:52:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.612+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1419eea0 con 0x7fec14100740 2024-03-19T01:52:47.617 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.613+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec14107ad0 con 0x7fec14100740 2024-03-19T01:52:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.614+0000 7fec11ffb640 1 -- 172.21.15.92:0/3622259959 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fec08044070 con 0x7fec14100740 2024-03-19T01:52:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.615+0000 7fec11ffb640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fec040a2880 0x7fec040a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.615+0000 7fec13fff640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fec040a2880 0x7fec040a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.615+0000 7fec11ffb640 1 -- 172.21.15.92:0/3622259959 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fec080f04b0 con 0x7fec14100740 2024-03-19T01:52:47.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.615+0000 7fec13fff640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fec040a2880 0x7fec040a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec14197fc0 tx=0x7fec00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.620+0000 7fec11ffb640 1 -- 172.21.15.92:0/3622259959 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fec080bcfa0 con 0x7fec14100740 2024-03-19T01:52:47.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.798+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec14062420 con 0x7fec14100740 2024-03-19T01:52:47.799 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.799+0000 7fec11ffb640 1 -- 172.21.15.92:0/3622259959 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22690 (secure 0 0 0) 0x7fec080c1040 con 0x7fec14100740 2024-03-19T01:52:47.800 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T01:52:47.800 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":87,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:51:40.581546+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fec040a2880 msgr2=0x7fec040a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fec040a2880 0x7fec040a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec14197fc0 tx=0x7fec00005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 msgr2=0x7fec14196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 0x7fec14196ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fec08000c00 tx=0x7fec0802c640 comp rx=0 tx=0).stop 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 shutdown_connections 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fec040a2880 0x7fec040a4d40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec141022f0 0x7fec1419e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec14101930 0x7fec14196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 --2- 172.21.15.92:0/3622259959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec14100740 0x7fec14196ab0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 >> 172.21.15.92:0/3622259959 conn(0x7fec140fbcb0 msgr2=0x7fec14104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 shutdown_connections 2024-03-19T01:52:47.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.801+0000 7fec1aff4640 1 -- 172.21.15.92:0/3622259959 wait complete. 2024-03-19T01:52:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:47 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3934657918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:47 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/4189952974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:47.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:47 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2376522501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:47.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-19T01:50:49.881618+0000', 'modified': '2024-03-19T01:50:56.972139+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24493, 'mds_1': 24476, 'mds_2': 24475, 'mds_3': 24457, 'mds_4': 14601}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14589': {'gid': 14589, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6839/1427243227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1427243227}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1427243227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14601': {'gid': 14601, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6841/133114375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6840', 'nonce': 133114375}, {'type': 'v1', 'addr': '172.21.15.92:6841', 'nonce': 133114375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6833/3429474924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3429474924}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3429474924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/2148025721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 2148025721}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 2148025721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/3252974913', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 3252974913}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 3252974913}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6837/1367568659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 1367568659}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 1367568659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.159:6835/610100257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 610100257}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 610100257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.159:6837/3903346711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 3903346711}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 3903346711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.144:6839/380126062', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 380126062}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 380126062}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.159:6839/2741545135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 2741545135}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 2741545135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24493, 'qdb_cluster': [24476, 24493, 24475, 24457, 14601]} 2024-03-19T01:52:47.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-19T01:52:47.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-19T01:52:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:47 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3934657918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:47 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/4189952974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:47.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:47 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2376522501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:47.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.923+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3475783358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc101910 msgr2=0x7fb3bc101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.923+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3475783358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc101910 0x7fb3bc101d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009900 tx=0x7fb3ac02bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:47.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.923+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3475783358 shutdown_connections 2024-03-19T01:52:47.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.923+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3475783358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc1022d0 0x7fb3bc108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.923+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3475783358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc101910 0x7fb3bc101d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.923+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3475783358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc100720 0x7fb3bc100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.923+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3475783358 >> 172.21.15.92:0/3475783358 conn(0x7fb3bc0fbcb0 msgr2=0x7fb3bc0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.923+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3475783358 shutdown_connections 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.923+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3475783358 wait complete. 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c3b05640 1 Processor -- start 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c3b05640 1 -- start start 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c3b05640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100720 0x7fb3bc196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c3b05640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc101910 0x7fb3bc196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c3b05640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1022d0 0x7fb3bc19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c3b05640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc10c3a0 con 0x7fb3bc100720 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c3b05640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc10c220 con 0x7fb3bc1022d0 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c3b05640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc10c520 con 0x7fb3bc101910 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c18a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100720 0x7fb3bc196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c18a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100720 0x7fb3bc196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53004/0 (socket says 172.21.15.92:53004) 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c18a1640 1 -- 172.21.15.92:0/3367453201 learned_addr learned my addr 172.21.15.92:0/3367453201 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c10a0640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc101910 0x7fb3bc196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c20a2640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1022d0 0x7fb3bc19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c18a1640 1 -- 172.21.15.92:0/3367453201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc101910 msgr2=0x7fb3bc196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc101910 0x7fb3bc196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c18a1640 1 -- 172.21.15.92:0/3367453201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1022d0 msgr2=0x7fb3bc19e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1022d0 0x7fb3bc19e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.924+0000 7fb3c18a1640 1 -- 172.21.15.92:0/3367453201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc19e690 con 0x7fb3bc100720 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.925+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100720 0x7fb3bc196a70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800ead0 tx=0x7fb3b800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.925+0000 7fb3b2ffd640 1 -- 172.21.15.92:0/3367453201 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b801c070 con 0x7fb3bc100720 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.925+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3367453201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc0689e0 con 0x7fb3bc100720 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.925+0000 7fb3b2ffd640 1 -- 172.21.15.92:0/3367453201 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b800bb70 con 0x7fb3bc100720 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.925+0000 7fb3b2ffd640 1 -- 172.21.15.92:0/3367453201 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8003d70 con 0x7fb3bc100720 2024-03-19T01:52:47.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.925+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3367453201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc068f00 con 0x7fb3bc100720 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.927+0000 7fb3b2ffd640 1 -- 172.21.15.92:0/3367453201 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb3b801f070 con 0x7fb3bc100720 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.927+0000 7fb3b2ffd640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2800 0x7fb3a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.928+0000 7fb3c10a0640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2800 0x7fb3a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.928+0000 7fb3b2ffd640 1 -- 172.21.15.92:0/3367453201 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb3b80d0100 con 0x7fb3bc100720 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.928+0000 7fb3c10a0640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2800 0x7fb3a80a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac02c7b0 tx=0x7fb3ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.928+0000 7fb3c67c5740 1 Processor -- start 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.928+0000 7fb3c67c5740 1 -- start start 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 0x55abcbbab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 0x55abcbbadfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55abcbb0ade0 con 0x55abcbba5ff0 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55abcbab3c80 con 0x55abcbbabb50 2024-03-19T01:52:47.935 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55abcbb06e80 con 0x55abcbba69e0 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c18a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c10a0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 0x55abcbbab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c18a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53006/0 (socket says 172.21.15.92:53006) 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c18a1640 1 -- 172.21.15.92:0/4072822020 learned_addr learned my addr 172.21.15.92:0/4072822020 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c20a2640 1 --2- 172.21.15.92:0/4072822020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 0x55abcbbadfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c18a1640 1 -- 172.21.15.92:0/4072822020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 msgr2=0x55abcbbab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/4072822020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 0x55abcbbab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c18a1640 1 -- 172.21.15.92:0/4072822020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 msgr2=0x55abcbbadfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/4072822020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 0x55abcbbadfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c18a1640 1 -- 172.21.15.92:0/4072822020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abcbac5fb0 con 0x55abcbba5ff0 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/4072822020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba6410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb3b809ca40 tx=0x7fb3b809d1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7a02f420cbcc44 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3a7fff640 1 -- 172.21.15.92:0/4072822020 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a0c00 con 0x55abcbba5ff0 2024-03-19T01:52:47.936 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3a7fff640 1 -- 172.21.15.92:0/4072822020 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b80a5250 con 0x55abcbba5ff0 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.929+0000 7fb3a7fff640 1 -- 172.21.15.92:0/4072822020 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b809c160 con 0x55abcbba5ff0 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.930+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4072822020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 msgr2=0x55abcbba6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.930+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/4072822020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba6410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb3b809ca40 tx=0x7fb3b809d1b0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.930+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4072822020 shutdown_connections 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.930+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/4072822020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 0x55abcbbadfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.930+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/4072822020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 0x55abcbbab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.930+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/4072822020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba6410 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.930+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4072822020 >> 172.21.15.92:0/4072822020 conn(0x55abcbba1560 msgr2=0x55abcbba1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.930+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4072822020 shutdown_connections 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.930+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4072822020 wait complete. 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c67c5740 1 -- start start 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c67c5740 1 Processor -- start 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbbb0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 0x55abcbbb0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.937 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbbb7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55abcbbaeb00 con 0x55abcbbabb50 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55abcbac6fc0 con 0x55abcbba69e0 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55abcbbaec80 con 0x55abcbba5ff0 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c67c5740 10 client.0 _ll_register_callbacks cb 0x55abcbb02a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c20a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbbb7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c20a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbbb7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53008/0 (socket says 172.21.15.92:53008) 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c20a2640 1 -- 172.21.15.92:0/3002848075 learned_addr learned my addr 172.21.15.92:0/3002848075 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbbb0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.931+0000 7fb3c10a0640 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 0x55abcbbb0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c20a2640 1 -- 172.21.15.92:0/3002848075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 msgr2=0x55abcbbb0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c20a2640 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbbb0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c20a2640 1 -- 172.21.15.92:0/3002848075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 msgr2=0x55abcbbb0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c20a2640 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 0x55abcbbb0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c20a2640 1 -- 172.21.15.92:0/3002848075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abcb81f2d0 con 0x55abcbbabb50 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbbb0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c20a2640 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbbb7b10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4012a00 tx=0x7fb3b4012ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc8d6e65807da5f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3a57fa640 1 -- 172.21.15.92:0/3002848075 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400f960 con 0x55abcbbabb50 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3a57fa640 1 -- 172.21.15.92:0/3002848075 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb3b400fb00 con 0x55abcbbabb50 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c67c5740 1 -- 172.21.15.92:0/3002848075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b80a5250 con 0x55abcbbabb50 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3a57fa640 1 -- 172.21.15.92:0/3002848075 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3b400fd00 con 0x55abcbbabb50 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c67c5740 10 client.15219 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c67c5740 1 -- 172.21.15.92:0/3002848075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abcb8e96b0 con 0x55abcbbabb50 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3a57fa640 1 -- 172.21.15.92:0/3002848075 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b80a5250 con 0x55abcbbabb50 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3a57fa640 1 -- 172.21.15.92:0/3002848075 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b401c420 con 0x55abcbbabb50 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c67c5740 10 client.15219 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c67c5740 10 client.15219 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-19T01:52:47.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c67c5740 20 client.15219 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x55abcbbc1850 0x55abcbbc2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c67c5740 4 client.15219 mds_command: new command op to 14601 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.932+0000 7fb3c67c5740 1 -- 172.21.15.92:0/3002848075 --> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abcbb18320 con 0x55abcbbc1850 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.933+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x55abcbbc1850 0x55abcbbc2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.933+0000 7fb3a57fa640 10 client.15219 ms_handle_connect on v2:172.21.15.92:6840/133114375 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.933+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x55abcbbc1850 0x55abcbbc2cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fec75f7c2b8226f7 server_cookie=5673913820b90176 in_seq=0 out_seq=0 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3a57fa640 1 -- 172.21.15.92:0/3002848075 <== mds.4 v2:172.21.15.92:6840/133114375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55abcbb18320 con 0x55abcbbc1850 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3a57fa640 10 client.15219 handle_command_reply: tid=0 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 client.15219 shutdown 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 -- 172.21.15.92:0/3002848075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 msgr2=0x55abcbbb7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.940 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbbb7b10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4012a00 tx=0x7fb3b4012ec0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 -- 172.21.15.92:0/3002848075 shutdown_connections 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x55abcbbc1850 0x55abcbbc2cc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3a57fa640 0 client.15219 ms_handle_reset on v2:172.21.15.92:6840/133114375 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbbb7b10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 0x55abcbbb0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/3002848075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbbb0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 -- 172.21.15.92:0/3002848075 >> 172.21.15.92:0/3002848075 conn(0x55abcbba1560 msgr2=0x55abcbba07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 -- 172.21.15.92:0/3002848075 shutdown_connections 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 1 -- 172.21.15.92:0/3002848075 wait complete. 2024-03-19T01:52:47.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.934+0000 7fb3c67c5740 20 client.15219 trim_cache size 0 max 16384 2024-03-19T01:52:47.957 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:47 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3934657918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T01:52:47.957 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:47 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/4189952974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T01:52:47.957 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:47 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2376522501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T01:52:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c67c5740 1 Processor -- start 2024-03-19T01:52:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c67c5740 1 -- start start 2024-03-19T01:52:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 0x55abcbba84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 0x55abcbbb7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55abcbbc1000 con 0x55abcbba5ff0 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55abcbbc16d0 con 0x55abcbbabb50 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55abcbbc1da0 con 0x55abcbba69e0 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c18a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c18a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53024/0 (socket says 172.21.15.92:53024) 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c18a1640 1 -- 172.21.15.92:0/4083336560 learned_addr learned my addr 172.21.15.92:0/4083336560 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c10a0640 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 0x55abcbba84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c20a2640 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 0x55abcbbb7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c18a1640 1 -- 172.21.15.92:0/4083336560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 msgr2=0x55abcbba84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 0x55abcbba84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c18a1640 1 -- 172.21.15.92:0/4083336560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 msgr2=0x55abcbbb7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 0x55abcbbb7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c18a1640 1 -- 172.21.15.92:0/4083336560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abcbbc0e30 con 0x55abcbba5ff0 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c20a2640 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 0x55abcbbb7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba7f80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb3b809d360 tx=0x7fb3b8099290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146c2dbc42ad0497 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.952+0000 7fb3a77fe640 1 -- 172.21.15.92:0/4083336560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a5250 con 0x55abcbba5ff0 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3a77fe640 1 -- 172.21.15.92:0/4083336560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b809adb0 con 0x55abcbba5ff0 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3a77fe640 1 -- 172.21.15.92:0/4083336560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80b5dd0 con 0x55abcbba5ff0 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4083336560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 msgr2=0x55abcbba7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba7f80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb3b809d360 tx=0x7fb3b8099290 comp rx=0 tx=0).stop 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4083336560 shutdown_connections 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbbabb50 0x55abcbbb7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba69e0 0x55abcbba84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/4083336560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbba5ff0 0x55abcbba7f80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4083336560 >> 172.21.15.92:0/4083336560 conn(0x55abcbba1560 msgr2=0x55abcbba1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4083336560 shutdown_connections 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 -- 172.21.15.92:0/4083336560 wait complete. 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 -- start start 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.953+0000 7fb3c67c5740 1 Processor -- start 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbba01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 0x55abcbba06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c67c5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbc255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55abcbbc2b40 con 0x55abcbbabb50 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55abcbbc2470 con 0x55abcbba69e0 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c67c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55abcbbc3210 con 0x55abcbba5ff0 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c67c5740 10 client.0 _ll_register_callbacks cb 0x7fb3b4009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c20a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbc255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c18a1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbba01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c20a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbc255b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53032/0 (socket says 172.21.15.92:53032) 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c20a2640 1 -- 172.21.15.92:0/1632777722 learned_addr learned my addr 172.21.15.92:0/1632777722 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c10a0640 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 0x55abcbba06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c18a1640 1 -- 172.21.15.92:0/1632777722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 msgr2=0x55abcbba06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 0x55abcbba06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c18a1640 1 -- 172.21.15.92:0/1632777722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 msgr2=0x55abcbc255b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbc255b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c18a1640 1 -- 172.21.15.92:0/1632777722 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abcbc25af0 con 0x55abcbba5ff0 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c20a2640 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbc255b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3c18a1640 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbba01a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55abcbbae760 tx=0x7fb3b8009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46df4060ba7b4cb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.954+0000 7fb3a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3a57fa640 1 -- 172.21.15.92:0/1632777722 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b809ab50 con 0x55abcbba5ff0 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3a57fa640 1 -- 172.21.15.92:0/1632777722 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb3b80b5b70 con 0x55abcbba5ff0 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c67c5740 1 -- 172.21.15.92:0/1632777722 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b809adb0 con 0x55abcbba5ff0 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3a57fa640 1 -- 172.21.15.92:0/1632777722 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3b80a0c00 con 0x55abcbba5ff0 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c67c5740 10 client.25111 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c67c5740 1 -- 172.21.15.92:0/1632777722 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abcbc26c70 con 0x55abcbba5ff0 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3a57fa640 1 -- 172.21.15.92:0/1632777722 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b809adb0 con 0x55abcbba5ff0 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3a57fa640 1 -- 172.21.15.92:0/1632777722 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b80a1b20 con 0x55abcbba5ff0 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c67c5740 10 client.25111 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c67c5740 10 client.25111 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c67c5740 20 client.25111 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x55abcbbc1850 0x55abcbc29e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c67c5740 4 client.25111 mds_command: new command op to 14601 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c67c5740 1 -- 172.21.15.92:0/1632777722 --> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] -- command(tid 0: {"prefix": "status"}) -- 0x55abcbb12670 con 0x55abcbbc1850 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.955+0000 7fb3c10a0640 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x55abcbbc1850 0x55abcbc29e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3a57fa640 10 client.25111 ms_handle_connect on v2:172.21.15.92:6840/133114375 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c10a0640 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x55abcbbc1850 0x55abcbc29e70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed05a8e269127e3c server_cookie=bf51cd07a9932dc0 in_seq=0 out_seq=0 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3a57fa640 1 -- 172.21.15.92:0/1632777722 <== mds.4 v2:172.21.15.92:6840/133114375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55abcbb12670 con 0x55abcbbc1850 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3a57fa640 10 client.25111 handle_command_reply: tid=0 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 client.25111 shutdown 2024-03-19T01:52:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 -- 172.21.15.92:0/1632777722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 msgr2=0x55abcbba01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbba01a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55abcbbae760 tx=0x7fb3b8009f20 comp rx=0 tx=0).stop 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 -- 172.21.15.92:0/1632777722 shutdown_connections 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.92:6840/133114375,v1:172.21.15.92:6841/133114375] conn(0x55abcbbc1850 0x55abcbc29e70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55abcbbabb50 0x55abcbc255b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3a57fa640 0 client.25111 ms_handle_reset on v2:172.21.15.92:6840/133114375 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abcbba69e0 0x55abcbba06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 --2- 172.21.15.92:0/1632777722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abcbba5ff0 0x55abcbba01a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 -- 172.21.15.92:0/1632777722 >> 172.21.15.92:0/1632777722 conn(0x55abcbba1560 msgr2=0x55abcbba7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 -- 172.21.15.92:0/1632777722 shutdown_connections 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 1 -- 172.21.15.92:0/1632777722 wait complete. 2024-03-19T01:52:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.956+0000 7fb3c67c5740 20 client.25111 trim_cache size 0 max 16384 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 4, 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 14601, 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 111.989309289, 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 126.719922135 2024-03-19T01:52:47.965 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2800 msgr2=0x7fb3a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2800 0x7fb3a80a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac02c7b0 tx=0x7fb3ac005fd0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100720 msgr2=0x7fb3bc196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100720 0x7fb3bc196a70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800ead0 tx=0x7fb3b800eed0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3367453201 shutdown_connections 2024-03-19T01:52:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1022d0 0x7fb3bc19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb3a80a2800 0x7fb3a80a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc101910 0x7fb3bc196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 --2- 172.21.15.92:0/3367453201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3bc100720 0x7fb3bc196a70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3367453201 >> 172.21.15.92:0/3367453201 conn(0x7fb3bc0fbcb0 msgr2=0x7fb3bc104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3367453201 shutdown_connections 2024-03-19T01:52:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:47.957+0000 7fb3c3b05640 1 -- 172.21.15.92:0/3367453201 wait complete. 2024-03-19T01:52:47.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-19T01:52:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.076+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/258019538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c280ffe70 msgr2=0x7f8c28100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.076+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/258019538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c280ffe70 0x7f8c28100290 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800aa10 tx=0x7f8c1802cf70 comp rx=0 tx=0).stop 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.076+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/258019538 shutdown_connections 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.076+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/258019538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c28101a10 0x7f8c28108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.076+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/258019538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c28101050 0x7f8c281014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.076+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/258019538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c280ffe70 0x7f8c28100290 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.076+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/258019538 >> 172.21.15.92:0/258019538 conn(0x7f8c280fb450 msgr2=0x7f8c280fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.076+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/258019538 shutdown_connections 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.076+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/258019538 wait complete. 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c2e9d4640 1 Processor -- start 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c2e9d4640 1 -- start start 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c2e9d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c280ffe70 0x7f8c28197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c2e9d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c28101050 0x7f8c281980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c2e9d4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c28101a10 0x7f8c2819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c2e9d4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2810b300 con 0x7f8c28101050 2024-03-19T01:52:48.079 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c2e9d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2810b180 con 0x7f8c280ffe70 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c2e9d4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2810b480 con 0x7f8c28101a10 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c28101050 0x7f8c281980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c28101050 0x7f8c281980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53046/0 (socket says 172.21.15.92:53046) 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c277fe640 1 -- 172.21.15.92:0/2979629914 learned_addr learned my addr 172.21.15.92:0/2979629914 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c2cf71640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c28101a10 0x7f8c2819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c27fff640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c280ffe70 0x7f8c28197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c277fe640 1 -- 172.21.15.92:0/2979629914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c28101a10 msgr2=0x7f8c2819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c277fe640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c28101a10 0x7f8c2819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c277fe640 1 -- 172.21.15.92:0/2979629914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c280ffe70 msgr2=0x7f8c28197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c277fe640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c280ffe70 0x7f8c28197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c277fe640 1 -- 172.21.15.92:0/2979629914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2819f7b0 con 0x7f8c28101050 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.077+0000 7f8c277fe640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c28101050 0x7f8c281980d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00e9f0 tx=0x7f8c1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.078+0000 7f8c257fa640 1 -- 172.21.15.92:0/2979629914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00cd80 con 0x7f8c28101050 2024-03-19T01:52:48.080 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.078+0000 7f8c257fa640 1 -- 172.21.15.92:0/2979629914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1c010910 con 0x7f8c28101050 2024-03-19T01:52:48.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.078+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/2979629914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2819faa0 con 0x7f8c28101050 2024-03-19T01:52:48.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.078+0000 7f8c257fa640 1 -- 172.21.15.92:0/2979629914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c0188f0 con 0x7f8c28101050 2024-03-19T01:52:48.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.078+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/2979629914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c2819fff0 con 0x7f8c28101050 2024-03-19T01:52:48.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c257fa640 1 -- 172.21.15.92:0/2979629914 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f8c1c010ac0 con 0x7f8c28101050 2024-03-19T01:52:48.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c315c1740 1 Processor -- start 2024-03-19T01:52:48.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c315c1740 1 -- start start 2024-03-19T01:52:48.087 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f209410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 0x55779f20e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f210fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c315c1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55779f16dde0 con 0x55779f208ff0 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c315c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55779f116c80 con 0x55779f2099e0 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c315c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55779f169e80 con 0x55779f20eb50 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c257fa640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8c100a2800 0x7f8c100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c2cf71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f209410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c2cf71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f209410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53048/0 (socket says 172.21.15.92:53048) 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c2cf71640 1 -- 172.21.15.92:0/99482649 learned_addr learned my addr 172.21.15.92:0/99482649 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c27fff640 1 --2- 172.21.15.92:0/99482649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 0x55779f20e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.080+0000 7f8c2cf71640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8c100a2800 0x7f8c100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c257fa640 1 -- 172.21.15.92:0/2979629914 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8c1c0cee00 con 0x7f8c28101050 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c277fe640 1 --2- 172.21.15.92:0/99482649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f210fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.088 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c27fff640 1 -- 172.21.15.92:0/99482649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 msgr2=0x55779f210fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c27fff640 1 --2- 172.21.15.92:0/99482649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f210fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c27fff640 1 -- 172.21.15.92:0/99482649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 msgr2=0x55779f209410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c27fff640 1 --2- 172.21.15.92:0/99482649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f209410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c27fff640 1 -- 172.21.15.92:0/99482649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55779f128fb0 con 0x55779f2099e0 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c2cf71640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8c100a2800 0x7f8c100a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009270 tx=0x7f8c20009bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c27fff640 1 --2- 172.21.15.92:0/99482649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 0x55779f20e4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800a340 tx=0x7f8c18005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b6daea76e30b5a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c0e7fc640 1 -- 172.21.15.92:0/99482649 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18043070 con 0x55779f2099e0 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c0e7fc640 1 -- 172.21.15.92:0/99482649 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c180046c0 con 0x55779f2099e0 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.081+0000 7f8c0e7fc640 1 -- 172.21.15.92:0/99482649 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c180359a0 con 0x55779f2099e0 2024-03-19T01:52:48.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 -- 172.21.15.92:0/99482649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 msgr2=0x55779f20e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 --2- 172.21.15.92:0/99482649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 0x55779f20e4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800a340 tx=0x7f8c18005d50 comp rx=0 tx=0).stop 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 -- 172.21.15.92:0/99482649 shutdown_connections 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 --2- 172.21.15.92:0/99482649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f210fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 --2- 172.21.15.92:0/99482649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 0x55779f20e4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 --2- 172.21.15.92:0/99482649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f209410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 -- 172.21.15.92:0/99482649 >> 172.21.15.92:0/99482649 conn(0x55779f204560 msgr2=0x55779f204960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 -- 172.21.15.92:0/99482649 shutdown_connections 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 -- 172.21.15.92:0/99482649 wait complete. 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 -- start start 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 Processor -- start 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f213470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f2099e0 0x55779f2139b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f21a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.082+0000 7f8c315c1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55779f211aa0 con 0x55779f2099e0 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c315c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55779f129fc0 con 0x55779f208ff0 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c315c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55779f211c20 con 0x55779f20eb50 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c315c1740 10 client.0 _ll_register_callbacks cb 0x55779f165a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f2099e0 0x55779f2139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f2099e0 0x55779f2139b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53054/0 (socket says 172.21.15.92:53054) 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f213470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c277fe640 1 -- 172.21.15.92:0/326847906 learned_addr learned my addr 172.21.15.92:0/326847906 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c2cf71640 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f21a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c27fff640 1 -- 172.21.15.92:0/326847906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 msgr2=0x55779f21a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c27fff640 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f21a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c27fff640 1 -- 172.21.15.92:0/326847906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f2099e0 msgr2=0x55779f2139b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c27fff640 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f2099e0 0x55779f2139b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c27fff640 1 -- 172.21.15.92:0/326847906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55779ee822d0 con 0x55779f208ff0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c277fe640 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f2099e0 0x55779f2139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c27fff640 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f213470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55779f20a450 tx=0x7f8c1802cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b91367c86dba647d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c077fe640 1 -- 172.21.15.92:0/326847906 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18043040 con 0x55779f208ff0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c077fe640 1 -- 172.21.15.92:0/326847906 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8c1802db20 con 0x55779f208ff0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c315c1740 1 -- 172.21.15.92:0/326847906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c180046c0 con 0x55779f208ff0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c077fe640 1 -- 172.21.15.92:0/326847906 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8c1802dd60 con 0x55779f208ff0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c315c1740 10 client.25076 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.083+0000 7f8c315c1740 1 -- 172.21.15.92:0/326847906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55779ef4c6b0 con 0x55779f208ff0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.084+0000 7f8c077fe640 1 -- 172.21.15.92:0/326847906 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c180046c0 con 0x55779f208ff0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.084+0000 7f8c077fe640 1 -- 172.21.15.92:0/326847906 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8c1800b040 con 0x55779f208ff0 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.084+0000 7f8c315c1740 10 client.25076 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.084+0000 7f8c315c1740 10 client.25076 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-19T01:52:48.092 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.084+0000 7f8c315c1740 20 client.25076 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.084+0000 7f8c315c1740 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x55779f21fd50 0x55779f2211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.084+0000 7f8c315c1740 4 client.25076 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.084+0000 7f8c315c1740 1 -- 172.21.15.92:0/326847906 --> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55779f17b320 con 0x55779f21fd50 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.084+0000 7f8c277fe640 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x55779f21fd50 0x55779f2211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.085+0000 7f8c277fe640 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x55779f21fd50 0x55779f2211c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e60f20343ddce44 server_cookie=8ee01e897c6b8e58 in_seq=0 out_seq=0 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.085+0000 7f8c077fe640 10 client.25076 ms_handle_connect on v2:172.21.15.159:6834/610100257 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.085+0000 7f8c077fe640 1 -- 172.21.15.92:0/326847906 <== mds.3 v2:172.21.15.159:6834/610100257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55779f17b320 con 0x55779f21fd50 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.085+0000 7f8c077fe640 10 client.25076 handle_command_reply: tid=0 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.085+0000 7f8c315c1740 1 client.25076 shutdown 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.085+0000 7f8c315c1740 1 -- 172.21.15.92:0/326847906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 msgr2=0x55779f213470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.085+0000 7f8c315c1740 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f213470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55779f20a450 tx=0x7f8c1802cf70 comp rx=0 tx=0).stop 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c315c1740 1 -- 172.21.15.92:0/326847906 shutdown_connections 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c315c1740 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x55779f21fd50 0x55779f2211c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c315c1740 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f21a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c077fe640 0 client.25076 ms_handle_reset on v2:172.21.15.159:6834/610100257 2024-03-19T01:52:48.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c315c1740 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f2099e0 0x55779f2139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c315c1740 1 --2- 172.21.15.92:0/326847906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f213470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c315c1740 1 -- 172.21.15.92:0/326847906 >> 172.21.15.92:0/326847906 conn(0x55779f204560 msgr2=0x55779f2037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c315c1740 1 -- 172.21.15.92:0/326847906 shutdown_connections 2024-03-19T01:52:48.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c315c1740 1 -- 172.21.15.92:0/326847906 wait complete. 2024-03-19T01:52:48.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.086+0000 7f8c315c1740 20 client.25076 trim_cache size 0 max 16384 2024-03-19T01:52:48.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c315c1740 1 Processor -- start 2024-03-19T01:52:48.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c315c1740 1 -- start start 2024-03-19T01:52:48.109 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f20ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 0x55779f20b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f21a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c315c1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55779f21f500 con 0x55779f208ff0 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c315c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55779f21fbd0 con 0x55779f2099e0 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c315c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55779f2202a0 con 0x55779f20eb50 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f20ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f20ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53062/0 (socket says 172.21.15.92:53062) 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c27fff640 1 -- 172.21.15.92:0/3267297669 learned_addr learned my addr 172.21.15.92:0/3267297669 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c2cf71640 1 --2- 172.21.15.92:0/3267297669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f21a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c277fe640 1 --2- 172.21.15.92:0/3267297669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 0x55779f20b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c27fff640 1 -- 172.21.15.92:0/3267297669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 msgr2=0x55779f21a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c27fff640 1 --2- 172.21.15.92:0/3267297669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f21a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c27fff640 1 -- 172.21.15.92:0/3267297669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 msgr2=0x55779f20b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c27fff640 1 --2- 172.21.15.92:0/3267297669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 0x55779f20b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c27fff640 1 -- 172.21.15.92:0/3267297669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55779f211b60 con 0x55779f208ff0 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c27fff640 1 --2- 172.21.15.92:0/3267297669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f20ae40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c18005700 tx=0x7f8c18005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2de64b141d35899 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.104+0000 7f8c0dffb640 1 -- 172.21.15.92:0/3267297669 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18043040 con 0x55779f208ff0 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c0dffb640 1 -- 172.21.15.92:0/3267297669 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c18057030 con 0x55779f208ff0 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c0dffb640 1 -- 172.21.15.92:0/3267297669 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c180046c0 con 0x55779f208ff0 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 -- 172.21.15.92:0/3267297669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 msgr2=0x55779f20ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3267297669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f20ae40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c18005700 tx=0x7f8c18005d50 comp rx=0 tx=0).stop 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 -- 172.21.15.92:0/3267297669 shutdown_connections 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3267297669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f20eb50 0x55779f21a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3267297669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f2099e0 0x55779f20b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3267297669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f208ff0 0x55779f20ae40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 -- 172.21.15.92:0/3267297669 >> 172.21.15.92:0/3267297669 conn(0x55779f204560 msgr2=0x55779f204940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 -- 172.21.15.92:0/3267297669 shutdown_connections 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 -- 172.21.15.92:0/3267297669 wait complete. 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.105+0000 7f8c315c1740 1 -- start start 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c315c1740 1 Processor -- start 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f215e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f2099e0 0x55779f2163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c315c1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f20eb50 0x55779f2193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c315c1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55779f221040 con 0x55779f20eb50 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c315c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55779f220970 con 0x55779f208ff0 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c315c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55779f221710 con 0x55779f2099e0 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c315c1740 10 client.0 _ll_register_callbacks cb 0x7f8c1800a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:48.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c2cf71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f20eb50 0x55779f2193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c2cf71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f20eb50 0x55779f2193f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53064/0 (socket says 172.21.15.92:53064) 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c2cf71640 1 -- 172.21.15.92:0/3393383835 learned_addr learned my addr 172.21.15.92:0/3393383835 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c277fe640 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f2099e0 0x55779f2163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c27fff640 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f215e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c2cf71640 1 -- 172.21.15.92:0/3393383835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f2099e0 msgr2=0x55779f2163d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c2cf71640 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f2099e0 0x55779f2163d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c2cf71640 1 -- 172.21.15.92:0/3393383835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 msgr2=0x55779f215e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c2cf71640 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f215e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c2cf71640 1 -- 172.21.15.92:0/3393383835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55779f2199e0 con 0x55779f20eb50 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c27fff640 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f215e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c2cf71640 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f20eb50 0x55779f2193f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8c20010740 tx=0x7f8c20009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc273c0e97162997 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.106+0000 7f8c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c077fe640 1 -- 172.21.15.92:0/3393383835 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20004450 con 0x55779f20eb50 2024-03-19T01:52:48.113 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c077fe640 1 -- 172.21.15.92:0/3393383835 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8c200045f0 con 0x55779f20eb50 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c315c1740 1 -- 172.21.15.92:0/3393383835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c18057030 con 0x55779f20eb50 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c077fe640 1 -- 172.21.15.92:0/3393383835 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8c200047f0 con 0x55779f20eb50 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c315c1740 10 client.15243 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c315c1740 1 -- 172.21.15.92:0/3393383835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55779f21ad50 con 0x55779f20eb50 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c077fe640 1 -- 172.21.15.92:0/3393383835 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c18057030 con 0x55779f20eb50 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c077fe640 1 -- 172.21.15.92:0/3393383835 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8c20020490 con 0x55779f20eb50 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c315c1740 10 client.15243 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c315c1740 10 client.15243 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c315c1740 20 client.15243 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x55779f28a510 0x55779f286950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c315c1740 4 client.15243 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c315c1740 1 -- 172.21.15.92:0/3393383835 --> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] -- command(tid 0: {"prefix": "status"}) -- 0x55779f175670 con 0x55779f28a510 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.107+0000 7f8c27fff640 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x55779f28a510 0x55779f286950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c077fe640 10 client.15243 ms_handle_connect on v2:172.21.15.159:6834/610100257 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c27fff640 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x55779f28a510 0x55779f286950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ff508742028719c server_cookie=bb1ee03de3af590d in_seq=0 out_seq=0 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c077fe640 1 -- 172.21.15.92:0/3393383835 <== mds.3 v2:172.21.15.159:6834/610100257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55779f175670 con 0x55779f28a510 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c077fe640 10 client.15243 handle_command_reply: tid=0 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 client.15243 shutdown 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 -- 172.21.15.92:0/3393383835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f20eb50 msgr2=0x55779f2193f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f20eb50 0x55779f2193f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8c20010740 tx=0x7f8c20009ae0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 -- 172.21.15.92:0/3393383835 shutdown_connections 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.159:6834/610100257,v1:172.21.15.159:6835/610100257] conn(0x55779f28a510 0x55779f286950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55779f20eb50 0x55779f2193f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55779f2099e0 0x55779f2163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c077fe640 0 client.15243 ms_handle_reset on v2:172.21.15.159:6834/610100257 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 --2- 172.21.15.92:0/3393383835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55779f208ff0 0x55779f215e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 -- 172.21.15.92:0/3393383835 >> 172.21.15.92:0/3393383835 conn(0x55779f204560 msgr2=0x55779f20ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 -- 172.21.15.92:0/3393383835 shutdown_connections 2024-03-19T01:52:48.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 1 -- 172.21.15.92:0/3393383835 wait complete. 2024-03-19T01:52:48.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.108+0000 7f8c315c1740 20 client.15243 trim_cache size 0 max 16384 2024-03-19T01:52:48.116 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:48.116 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:48.116 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 3, 2024-03-19T01:52:48.116 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24457, 2024-03-19T01:52:48.116 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:48.116 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:48.117 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:48.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 113.11401710600001, 2024-03-19T01:52:48.117 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:48.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:48.117 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:48.117 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 141.25009947699999 2024-03-19T01:52:48.117 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8c100a2800 msgr2=0x7f8c100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8c100a2800 0x7f8c100a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009270 tx=0x7f8c20009bd0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c28101050 msgr2=0x7f8c281980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c28101050 0x7f8c281980d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00e9f0 tx=0x7f8c1c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/2979629914 shutdown_connections 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8c100a2800 0x7f8c100a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c28101a10 0x7f8c2819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c28101050 0x7f8c281980d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 --2- 172.21.15.92:0/2979629914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c280ffe70 0x7f8c28197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/2979629914 >> 172.21.15.92:0/2979629914 conn(0x7f8c280fb450 msgr2=0x7f8c281043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/2979629914 shutdown_connections 2024-03-19T01:52:48.119 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.109+0000 7f8c2e9d4640 1 -- 172.21.15.92:0/2979629914 wait complete. 2024-03-19T01:52:48.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.223+0000 7f0252d53640 1 -- 172.21.15.92:0/654742099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f024c104c50 msgr2=0x7f024c1050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.223+0000 7f0252d53640 1 --2- 172.21.15.92:0/654742099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f024c104c50 0x7f024c1050d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0238009900 tx=0x7f023802bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.223+0000 7f0252d53640 1 -- 172.21.15.92:0/654742099 shutdown_connections 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.223+0000 7f0252d53640 1 --2- 172.21.15.92:0/654742099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f024c105610 0x7f024c10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.223+0000 7f0252d53640 1 --2- 172.21.15.92:0/654742099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f024c104c50 0x7f024c1050d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.223+0000 7f0252d53640 1 --2- 172.21.15.92:0/654742099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f024c103a60 0x7f024c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.223+0000 7f0252d53640 1 -- 172.21.15.92:0/654742099 >> 172.21.15.92:0/654742099 conn(0x7f024c0fefd0 msgr2=0x7f024c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.223+0000 7f0252d53640 1 -- 172.21.15.92:0/654742099 shutdown_connections 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.223+0000 7f0252d53640 1 -- 172.21.15.92:0/654742099 wait complete. 2024-03-19T01:52:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0252d53640 1 Processor -- start 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0252d53640 1 -- start start 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0252d53640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f024c103a60 0x7f024c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0252d53640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f024c104c50 0x7f024c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0252d53640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f024c105610 0x7f024c1a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0252d53640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f024c10f710 con 0x7f024c103a60 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0252d53640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f024c10f590 con 0x7f024c104c50 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0252d53640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f024c10f890 con 0x7f024c105610 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0250aef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f024c103a60 0x7f024c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0250aef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f024c103a60 0x7f024c199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53080/0 (socket says 172.21.15.92:53080) 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0250aef640 1 -- 172.21.15.92:0/768560348 learned_addr learned my addr 172.21.15.92:0/768560348 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f02512f0640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f024c105610 0x7f024c1a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f024bfff640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f024c104c50 0x7f024c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0250aef640 1 -- 172.21.15.92:0/768560348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f024c105610 msgr2=0x7f024c1a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0250aef640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f024c105610 0x7f024c1a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.227 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0250aef640 1 -- 172.21.15.92:0/768560348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f024c104c50 msgr2=0x7f024c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0250aef640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f024c104c50 0x7f024c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.224+0000 7f0250aef640 1 -- 172.21.15.92:0/768560348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c1a1950 con 0x7f024c103a60 2024-03-19T01:52:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.225+0000 7f0250aef640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f024c103a60 0x7f024c199d40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f024000e9c0 tx=0x7f024000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.225+0000 7f0249ffb640 1 -- 172.21.15.92:0/768560348 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024000cd80 con 0x7f024c103a60 2024-03-19T01:52:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.225+0000 7f0249ffb640 1 -- 172.21.15.92:0/768560348 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02400035f0 con 0x7f024c103a60 2024-03-19T01:52:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.225+0000 7f0249ffb640 1 -- 172.21.15.92:0/768560348 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240010690 con 0x7f024c103a60 2024-03-19T01:52:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.225+0000 7f0252d53640 1 -- 172.21.15.92:0/768560348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024c1a1c40 con 0x7f024c103a60 2024-03-19T01:52:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.225+0000 7f0252d53640 1 -- 172.21.15.92:0/768560348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f024c1a2140 con 0x7f024c103a60 2024-03-19T01:52:48.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.227+0000 7f0249ffb640 1 -- 172.21.15.92:0/768560348 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f02400037a0 con 0x7f024c103a60 2024-03-19T01:52:48.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0255a1b740 1 Processor -- start 2024-03-19T01:52:48.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0255a1b740 1 -- start start 2024-03-19T01:52:48.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0249ffb640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f023c0a2800 0x7f023c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293ceff0 0x562b293cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293cf9e0 0x562b293d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293d4b50 0x562b293d6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f02512f0640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f023c0a2800 0x7f023c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0255a1b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b29333de0 con 0x562b293ceff0 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0255a1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562b292dcc80 con 0x562b293d4b50 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0255a1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b2932fe80 con 0x562b293cf9e0 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0249ffb640 1 -- 172.21.15.92:0/768560348 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f02400cee70 con 0x7f024c103a60 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f024bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293cf9e0 0x562b293d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0250aef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293ceff0 0x562b293cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f024bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293cf9e0 0x562b293d44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:39208/0 (socket says 172.21.15.92:39208) 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f024bfff640 1 -- 172.21.15.92:0/4030876716 learned_addr learned my addr 172.21.15.92:0/4030876716 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f02512f0640 1 --2- 172.21.15.92:0/4030876716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293d4b50 0x562b293d6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0250aef640 1 -- 172.21.15.92:0/4030876716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293cf9e0 msgr2=0x562b293d44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0250aef640 1 --2- 172.21.15.92:0/4030876716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293cf9e0 0x562b293d44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0250aef640 1 -- 172.21.15.92:0/4030876716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293d4b50 msgr2=0x562b293d6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f02512f0640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f023c0a2800 0x7f023c0a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0244009fd0 tx=0x7f0244013b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0250aef640 1 --2- 172.21.15.92:0/4030876716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293d4b50 0x562b293d6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f0250aef640 1 -- 172.21.15.92:0/4030876716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b292eefb0 con 0x562b293ceff0 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.228+0000 7f024bfff640 1 --2- 172.21.15.92:0/4030876716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293cf9e0 0x562b293d44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0250aef640 1 --2- 172.21.15.92:0/4030876716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293ceff0 0x562b293cf410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f024000c6a0 tx=0x7f024009b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e45135178dca789 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0236ffd640 1 -- 172.21.15.92:0/4030876716 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024009b820 con 0x562b293ceff0 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0236ffd640 1 -- 172.21.15.92:0/4030876716 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024009f970 con 0x562b293ceff0 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0236ffd640 1 -- 172.21.15.92:0/4030876716 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024009cb50 con 0x562b293ceff0 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0255a1b740 1 -- 172.21.15.92:0/4030876716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293ceff0 msgr2=0x562b293cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0255a1b740 1 --2- 172.21.15.92:0/4030876716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293ceff0 0x562b293cf410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f024000c6a0 tx=0x7f024009b180 comp rx=0 tx=0).stop 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0255a1b740 1 -- 172.21.15.92:0/4030876716 shutdown_connections 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0255a1b740 1 --2- 172.21.15.92:0/4030876716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293d4b50 0x562b293d6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0255a1b740 1 --2- 172.21.15.92:0/4030876716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293cf9e0 0x562b293d44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.229+0000 7f0255a1b740 1 --2- 172.21.15.92:0/4030876716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293ceff0 0x562b293cf410 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 -- 172.21.15.92:0/4030876716 >> 172.21.15.92:0/4030876716 conn(0x562b293ca560 msgr2=0x562b293ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 -- 172.21.15.92:0/4030876716 shutdown_connections 2024-03-19T01:52:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 -- 172.21.15.92:0/4030876716 wait complete. 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 -- start start 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 Processor -- start 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293d9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b293d7aa0 con 0x562b293cf9e0 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562b292effc0 con 0x562b293ceff0 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b293d7c20 con 0x562b293d4b50 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0255a1b740 10 client.0 _ll_register_callbacks cb 0x562b2932ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f024bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f024bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293d99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53102/0 (socket says 172.21.15.92:53102) 2024-03-19T01:52:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f024bfff640 1 -- 172.21.15.92:0/1405256568 learned_addr learned my addr 172.21.15.92:0/1405256568 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f02512f0640 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.230+0000 7f0250aef640 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293d9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f024bfff640 1 -- 172.21.15.92:0/1405256568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 msgr2=0x562b293e0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f024bfff640 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293e0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f024bfff640 1 -- 172.21.15.92:0/1405256568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 msgr2=0x562b293d9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f024bfff640 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293d9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f024bfff640 1 -- 172.21.15.92:0/1405256568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b290482d0 con 0x562b293cf9e0 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f02512f0640 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f0250aef640 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293d9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f024bfff640 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293d99c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0238036040 tx=0x7f0238004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dad677f90183429 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f022bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f022bfff640 1 -- 172.21.15.92:0/1405256568 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238005d50 con 0x562b293cf9e0 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f022bfff640 1 -- 172.21.15.92:0/1405256568 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f02380046c0 con 0x562b293cf9e0 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f0255a1b740 1 -- 172.21.15.92:0/1405256568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f024009f970 con 0x562b293cf9e0 2024-03-19T01:52:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f022bfff640 1 -- 172.21.15.92:0/1405256568 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02380048c0 con 0x562b293cf9e0 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f0255a1b740 10 client.15255 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f0255a1b740 1 -- 172.21.15.92:0/1405256568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b291126b0 con 0x562b293cf9e0 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f022bfff640 1 -- 172.21.15.92:0/1405256568 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024009f970 con 0x562b293cf9e0 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.231+0000 7f022bfff640 1 -- 172.21.15.92:0/1405256568 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0238003860 con 0x562b293cf9e0 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.232+0000 7f0255a1b740 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.232+0000 7f0255a1b740 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.232+0000 7f0255a1b740 20 client.15255 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.232+0000 7f0255a1b740 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x562b293e6230 0x562b293e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.232+0000 7f0255a1b740 4 client.15255 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.232+0000 7f0255a1b740 1 -- 172.21.15.92:0/1405256568 --> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b29341320 con 0x562b293e6230 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.232+0000 7f0250aef640 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x562b293e6230 0x562b293e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.232+0000 7f0250aef640 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x562b293e6230 0x562b293e76a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7923704a26ee40f server_cookie=ffb760084129f465 in_seq=0 out_seq=0 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.232+0000 7f022bfff640 10 client.15255 ms_handle_connect on v2:172.21.15.159:6836/3903346711 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f022bfff640 1 -- 172.21.15.92:0/1405256568 <== mds.2 v2:172.21.15.159:6836/3903346711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562b29341320 con 0x562b293e6230 2024-03-19T01:52:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f022bfff640 10 client.15255 handle_command_reply: tid=0 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 client.15255 shutdown 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 -- 172.21.15.92:0/1405256568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 msgr2=0x562b293d99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293d99c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0238036040 tx=0x7f0238004270 comp rx=0 tx=0).stop 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 -- 172.21.15.92:0/1405256568 shutdown_connections 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x562b293e6230 0x562b293e76a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293d99c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 --2- 172.21.15.92:0/1405256568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293d9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f022bfff640 0 client.15255 ms_handle_reset on v2:172.21.15.159:6836/3903346711 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 -- 172.21.15.92:0/1405256568 >> 172.21.15.92:0/1405256568 conn(0x562b293ca560 msgr2=0x562b293c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 -- 172.21.15.92:0/1405256568 shutdown_connections 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 1 -- 172.21.15.92:0/1405256568 wait complete. 2024-03-19T01:52:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.233+0000 7f0255a1b740 20 client.15255 trim_cache size 0 max 16384 2024-03-19T01:52:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f0255a1b740 1 Processor -- start 2024-03-19T01:52:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f0255a1b740 1 -- start start 2024-03-19T01:52:48.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293ceff0 0x562b293d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293cf9e0 0x562b293e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293d4b50 0x562b293d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f0255a1b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b293e59e0 con 0x562b293d4b50 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f0255a1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562b293e60b0 con 0x562b293cf9e0 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f0255a1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b293e6780 con 0x562b293ceff0 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f02512f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293d4b50 0x562b293d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f02512f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293d4b50 0x562b293d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53114/0 (socket says 172.21.15.92:53114) 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f02512f0640 1 -- 172.21.15.92:0/841948242 learned_addr learned my addr 172.21.15.92:0/841948242 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f0250aef640 1 --2- 172.21.15.92:0/841948242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293ceff0 0x562b293d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.258+0000 7f024bfff640 1 --2- 172.21.15.92:0/841948242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293cf9e0 0x562b293e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.259+0000 7f02512f0640 1 -- 172.21.15.92:0/841948242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293ceff0 msgr2=0x562b293d4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.259+0000 7f02512f0640 1 --2- 172.21.15.92:0/841948242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293ceff0 0x562b293d4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.259+0000 7f02512f0640 1 -- 172.21.15.92:0/841948242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293cf9e0 msgr2=0x562b293e09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.259+0000 7f02512f0640 1 --2- 172.21.15.92:0/841948242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293cf9e0 0x562b293e09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.259+0000 7f02512f0640 1 -- 172.21.15.92:0/841948242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b293cfdc0 con 0x562b293d4b50 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.259+0000 7f02512f0640 1 --2- 172.21.15.92:0/841948242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293d4b50 0x562b293d0f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0244020460 tx=0x7f024401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc957a2d80bd6848 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.259+0000 7f02367fc640 1 -- 172.21.15.92:0/841948242 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244012030 con 0x562b293d4b50 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.259+0000 7f02367fc640 1 -- 172.21.15.92:0/841948242 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0244003e50 con 0x562b293d4b50 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.259+0000 7f02367fc640 1 -- 172.21.15.92:0/841948242 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244010430 con 0x562b293d4b50 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 -- 172.21.15.92:0/841948242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293d4b50 msgr2=0x562b293d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 --2- 172.21.15.92:0/841948242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293d4b50 0x562b293d0f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0244020460 tx=0x7f024401f000 comp rx=0 tx=0).stop 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 -- 172.21.15.92:0/841948242 shutdown_connections 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 --2- 172.21.15.92:0/841948242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293d4b50 0x562b293d0f40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 --2- 172.21.15.92:0/841948242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293cf9e0 0x562b293e09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 --2- 172.21.15.92:0/841948242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293ceff0 0x562b293d4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 -- 172.21.15.92:0/841948242 >> 172.21.15.92:0/841948242 conn(0x562b293ca560 msgr2=0x562b293ca940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 -- 172.21.15.92:0/841948242 shutdown_connections 2024-03-19T01:52:48.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 -- 172.21.15.92:0/841948242 wait complete. 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 -- start start 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 Processor -- start 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293dec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293df680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b293e7520 con 0x562b293cf9e0 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562b293e6e50 con 0x562b293ceff0 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.260+0000 7f0255a1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b293e7bf0 con 0x562b293d4b50 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f0255a1b740 10 client.0 _ll_register_callbacks cb 0x7f0238009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f024bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f0250aef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f024bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293df140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53128/0 (socket says 172.21.15.92:53128) 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f024bfff640 1 -- 172.21.15.92:0/2339634683 learned_addr learned my addr 172.21.15.92:0/2339634683 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f02512f0640 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293df680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f024bfff640 1 -- 172.21.15.92:0/2339634683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 msgr2=0x562b293df680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f024bfff640 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293df680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f024bfff640 1 -- 172.21.15.92:0/2339634683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 msgr2=0x562b293dec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f024bfff640 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293dec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f024bfff640 1 -- 172.21.15.92:0/2339634683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b293dfeb0 con 0x562b293cf9e0 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f0250aef640 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f02512f0640 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293df680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f024bfff640 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293df140 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0238009850 tx=0x7f023802cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a1d3cac4298a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f022bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f022bfff640 1 -- 172.21.15.92:0/2339634683 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238005d50 con 0x562b293cf9e0 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f022bfff640 1 -- 172.21.15.92:0/2339634683 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0238004eb0 con 0x562b293cf9e0 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f0255a1b740 1 -- 172.21.15.92:0/2339634683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0244003e50 con 0x562b293cf9e0 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f022bfff640 1 -- 172.21.15.92:0/2339634683 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0238034440 con 0x562b293cf9e0 2024-03-19T01:52:48.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f0255a1b740 10 client.15261 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f0255a1b740 1 -- 172.21.15.92:0/2339634683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b2944c190 con 0x562b293cf9e0 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.261+0000 7f022bfff640 1 -- 172.21.15.92:0/2339634683 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0244003e50 con 0x562b293cf9e0 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f022bfff640 1 -- 172.21.15.92:0/2339634683 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f023804b860 con 0x562b293cf9e0 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 10 client.15261 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 20 client.15261 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x562b29450df0 0x562b2944d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 4 client.15261 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 1 -- 172.21.15.92:0/2339634683 --> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] -- command(tid 0: {"prefix": "status"}) -- 0x562b2933b670 con 0x562b29450df0 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0250aef640 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x562b29450df0 0x562b2944d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f022bfff640 10 client.15261 ms_handle_connect on v2:172.21.15.159:6836/3903346711 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0250aef640 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x562b29450df0 0x562b2944d230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f04d09bf6245987b server_cookie=62344d6226b9314e in_seq=0 out_seq=0 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f022bfff640 1 -- 172.21.15.92:0/2339634683 <== mds.2 v2:172.21.15.159:6836/3903346711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562b2933b670 con 0x562b29450df0 2024-03-19T01:52:48.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f022bfff640 10 client.15261 handle_command_reply: tid=0 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 1 client.15261 shutdown 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 1 -- 172.21.15.92:0/2339634683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 msgr2=0x562b293df140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293df140 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0238009850 tx=0x7f023802cd20 comp rx=0 tx=0).stop 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 1 -- 172.21.15.92:0/2339634683 shutdown_connections 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.262+0000 7f0255a1b740 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.159:6836/3903346711,v1:172.21.15.159:6837/3903346711] conn(0x562b29450df0 0x562b2944d230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0255a1b740 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b293d4b50 0x562b293df680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f022bfff640 0 client.15261 ms_handle_reset on v2:172.21.15.159:6836/3903346711 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0255a1b740 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b293cf9e0 0x562b293df140 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0255a1b740 1 --2- 172.21.15.92:0/2339634683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b293ceff0 0x562b293dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0255a1b740 1 -- 172.21.15.92:0/2339634683 >> 172.21.15.92:0/2339634683 conn(0x562b293ca560 msgr2=0x562b293d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0255a1b740 1 -- 172.21.15.92:0/2339634683 shutdown_connections 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0255a1b740 1 -- 172.21.15.92:0/2339634683 wait complete. 2024-03-19T01:52:48.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0255a1b740 20 client.15261 trim_cache size 0 max 16384 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 2, 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24475, 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 114.274924947, 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:48.271 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:48.272 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 138.498631808 2024-03-19T01:52:48.272 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:48.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 -- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f023c0a2800 msgr2=0x7f023c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f023c0a2800 0x7f023c0a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0244009fd0 tx=0x7f0244013b80 comp rx=0 tx=0).stop 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 -- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f024c103a60 msgr2=0x7f024c199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f024c103a60 0x7f024c199d40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f024000e9c0 tx=0x7f024000ee80 comp rx=0 tx=0).stop 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 -- 172.21.15.92:0/768560348 shutdown_connections 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f023c0a2800 0x7f023c0a4cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f024c105610 0x7f024c1a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f024c104c50 0x7f024c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 --2- 172.21.15.92:0/768560348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f024c103a60 0x7f024c199d40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 -- 172.21.15.92:0/768560348 >> 172.21.15.92:0/768560348 conn(0x7f024c0fefd0 msgr2=0x7f024c107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 -- 172.21.15.92:0/768560348 shutdown_connections 2024-03-19T01:52:48.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.263+0000 7f0252d53640 1 -- 172.21.15.92:0/768560348 wait complete. 2024-03-19T01:52:48.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-19T01:52:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 -- 172.21.15.92:0/2189481375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c101950 msgr2=0x7f397c101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 --2- 172.21.15.92:0/2189481375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c101950 0x7f397c101dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3970009860 tx=0x7f397002bb70 comp rx=0 tx=0).stop 2024-03-19T01:52:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 -- 172.21.15.92:0/2189481375 shutdown_connections 2024-03-19T01:52:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 --2- 172.21.15.92:0/2189481375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102310 0x7f397c108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 --2- 172.21.15.92:0/2189481375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c101950 0x7f397c101dd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 --2- 172.21.15.92:0/2189481375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397c100760 0x7f397c100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 -- 172.21.15.92:0/2189481375 >> 172.21.15.92:0/2189481375 conn(0x7f397c0fbcd0 msgr2=0x7f397c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 -- 172.21.15.92:0/2189481375 shutdown_connections 2024-03-19T01:52:48.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 -- 172.21.15.92:0/2189481375 wait complete. 2024-03-19T01:52:48.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.379+0000 7f398275b640 1 Processor -- start 2024-03-19T01:52:48.381 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f398275b640 1 -- start start 2024-03-19T01:52:48.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f398275b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397c100760 0x7f397c196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.394 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f398275b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c101950 0x7f397c197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f398275b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102310 0x7f397c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f398275b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10c3c0 con 0x7f397c102310 2024-03-19T01:52:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f398275b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10c240 con 0x7f397c100760 2024-03-19T01:52:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f398275b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f397c10c540 con 0x7f397c101950 2024-03-19T01:52:48.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f3980cf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102310 0x7f397c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c101950 0x7f397c197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f3980cf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102310 0x7f397c19e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53148/0 (socket says 172.21.15.92:53148) 2024-03-19T01:52:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f397bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397c100760 0x7f397c196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f3980cf8640 1 -- 172.21.15.92:0/2712846162 learned_addr learned my addr 172.21.15.92:0/2712846162 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f3980cf8640 1 -- 172.21.15.92:0/2712846162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c101950 msgr2=0x7f397c197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f3980cf8640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c101950 0x7f397c197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f3980cf8640 1 -- 172.21.15.92:0/2712846162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397c100760 msgr2=0x7f397c196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f3980cf8640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397c100760 0x7f397c196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f3980cf8640 1 -- 172.21.15.92:0/2712846162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397c19e740 con 0x7f397c102310 2024-03-19T01:52:48.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f397b7fe640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c101950 0x7f397c197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.380+0000 7f397bfff640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397c100760 0x7f397c196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.381+0000 7f3980cf8640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102310 0x7f397c19e0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f397400efc0 tx=0x7f397400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.381+0000 7f39797fa640 1 -- 172.21.15.92:0/2712846162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974003970 con 0x7f397c102310 2024-03-19T01:52:48.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.381+0000 7f398275b640 1 -- 172.21.15.92:0/2712846162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397c19ea30 con 0x7f397c102310 2024-03-19T01:52:48.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.381+0000 7f39797fa640 1 -- 172.21.15.92:0/2712846162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3974004020 con 0x7f397c102310 2024-03-19T01:52:48.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.381+0000 7f39797fa640 1 -- 172.21.15.92:0/2712846162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397400b9f0 con 0x7f397c102310 2024-03-19T01:52:48.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.381+0000 7f398275b640 1 -- 172.21.15.92:0/2712846162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397c19ef50 con 0x7f397c102310 2024-03-19T01:52:48.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.383+0000 7f39797fa640 1 -- 172.21.15.92:0/2712846162 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f397401d020 con 0x7f397c102310 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.383+0000 7f39797fa640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39640a2800 0x7f39640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.383+0000 7f397bfff640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39640a2800 0x7f39640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.383+0000 7f39797fa640 1 -- 172.21.15.92:0/2712846162 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f39740ce8d0 con 0x7f397c102310 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397bfff640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39640a2800 0x7f39640a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f397c1017b0 tx=0x7f396c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f3985423740 1 Processor -- start 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f3985423740 1 -- start start 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f3985423740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 0x565077749410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f3985423740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650777499e0 0x56507774e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f3985423740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 0x565077750fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.399 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f3985423740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5650776adde0 con 0x5650777499e0 2024-03-19T01:52:48.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f3985423740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565077656c80 con 0x565077748ff0 2024-03-19T01:52:48.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f3985423740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650776a9e80 con 0x56507774eb50 2024-03-19T01:52:48.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650777499e0 0x56507774e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650777499e0 0x56507774e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53156/0 (socket says 172.21.15.92:53156) 2024-03-19T01:52:48.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397b7fe640 1 -- 172.21.15.92:0/3536698504 learned_addr learned my addr 172.21.15.92:0/3536698504 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397bfff640 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 0x565077749410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f3980cf8640 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 0x565077750fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.400 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397b7fe640 1 -- 172.21.15.92:0/3536698504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 msgr2=0x565077750fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397b7fe640 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 0x565077750fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397b7fe640 1 -- 172.21.15.92:0/3536698504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 msgr2=0x565077749410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397b7fe640 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 0x565077749410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397b7fe640 1 -- 172.21.15.92:0/3536698504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565077668fb0 con 0x5650777499e0 2024-03-19T01:52:48.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.384+0000 7f397bfff640 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 0x565077749410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:48.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f397b7fe640 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650777499e0 0x56507774e4e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3970009860 tx=0x7f3970034ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374f23c47c2ea055 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.401 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f39627fc640 1 -- 172.21.15.92:0/3536698504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970039040 con 0x5650777499e0 2024-03-19T01:52:48.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f39627fc640 1 -- 172.21.15.92:0/3536698504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3970034640 con 0x5650777499e0 2024-03-19T01:52:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f39627fc640 1 -- 172.21.15.92:0/3536698504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397002cb20 con 0x5650777499e0 2024-03-19T01:52:48.862 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f3985423740 1 -- 172.21.15.92:0/3536698504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650777499e0 msgr2=0x56507774e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f3985423740 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650777499e0 0x56507774e4e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3970009860 tx=0x7f3970034ad0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f3985423740 1 -- 172.21.15.92:0/3536698504 shutdown_connections 2024-03-19T01:52:48.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f3985423740 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 0x565077750fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f3985423740 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650777499e0 0x56507774e4e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.864 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.385+0000 7f3985423740 1 --2- 172.21.15.92:0/3536698504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 0x565077749410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.866 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 -- 172.21.15.92:0/3536698504 >> 172.21.15.92:0/3536698504 conn(0x565077744560 msgr2=0x565077744960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.867 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 -- 172.21.15.92:0/3536698504 shutdown_connections 2024-03-19T01:52:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 -- 172.21.15.92:0/3536698504 wait complete. 2024-03-19T01:52:48.868 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 -- start start 2024-03-19T01:52:48.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 Processor -- start 2024-03-19T01:52:48.869 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 0x565077755bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56507774eb50 0x56507775cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565077751aa0 con 0x565077748ff0 2024-03-19T01:52:48.870 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565077669fc0 con 0x56507774eb50 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565077751c20 con 0x5650777499e0 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3985423740 10 client.0 _ll_register_callbacks cb 0x5650776a5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f397bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f397bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53172/0 (socket says 172.21.15.92:53172) 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f397bfff640 1 -- 172.21.15.92:0/2813103663 learned_addr learned my addr 172.21.15.92:0/2813103663 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f397b7fe640 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 0x565077755bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.386+0000 7f3980cf8640 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56507774eb50 0x56507775cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f397bfff640 1 -- 172.21.15.92:0/2813103663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 msgr2=0x565077755bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f397bfff640 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 0x565077755bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f397bfff640 1 -- 172.21.15.92:0/2813103663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56507774eb50 msgr2=0x56507775cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f397bfff640 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56507774eb50 0x56507775cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f397bfff640 1 -- 172.21.15.92:0/2813103663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650773c22d0 con 0x565077748ff0 2024-03-19T01:52:48.871 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3980cf8640 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56507774eb50 0x56507775cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f397bfff640 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f396c003690 tx=0x7f396c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b156640b782d90eb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f395b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f395b7fe640 1 -- 172.21.15.92:0/2813103663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c003e00 con 0x565077748ff0 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f395b7fe640 1 -- 172.21.15.92:0/2813103663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f396c00f690 con 0x565077748ff0 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3985423740 1 -- 172.21.15.92:0/2813103663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3970034640 con 0x565077748ff0 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f395b7fe640 1 -- 172.21.15.92:0/2813103663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f396c00f890 con 0x565077748ff0 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3985423740 10 client.15273 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3985423740 1 -- 172.21.15.92:0/2813103663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56507748c6b0 con 0x565077748ff0 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f395b7fe640 1 -- 172.21.15.92:0/2813103663 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3970034640 con 0x565077748ff0 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f395b7fe640 1 -- 172.21.15.92:0/2813103663 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f396c031bd0 con 0x565077748ff0 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3985423740 10 client.15273 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3985423740 10 client.15273 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:52:48.872 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3985423740 20 client.15273 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3985423740 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x5650777624a0 0x565077763910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3985423740 4 client.15273 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.387+0000 7f3985423740 1 -- 172.21.15.92:0/2813103663 --> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650776bb320 con 0x5650777624a0 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.388+0000 7f397b7fe640 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x5650777624a0 0x565077763910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.388+0000 7f395b7fe640 10 client.15273 ms_handle_connect on v2:172.21.15.144:6838/380126062 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.388+0000 7f397b7fe640 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x5650777624a0 0x565077763910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5707675b720912d1 server_cookie=828a9ef62b6c268e in_seq=0 out_seq=0 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f395b7fe640 1 -- 172.21.15.92:0/2813103663 <== mds.1 v2:172.21.15.144:6838/380126062 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5650776bb320 con 0x5650777624a0 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f395b7fe640 10 client.15273 handle_command_reply: tid=0 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 client.15273 shutdown 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 -- 172.21.15.92:0/2813103663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 msgr2=0x565077755690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f396c003690 tx=0x7f396c004730 comp rx=0 tx=0).stop 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 -- 172.21.15.92:0/2813103663 shutdown_connections 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x5650777624a0 0x565077763910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56507774eb50 0x56507775cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.873 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 0x565077755bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f395b7fe640 0 client.15273 ms_handle_reset on v2:172.21.15.144:6838/380126062 2024-03-19T01:52:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 --2- 172.21.15.92:0/2813103663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755690 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 -- 172.21.15.92:0/2813103663 >> 172.21.15.92:0/2813103663 conn(0x565077744560 msgr2=0x5650777437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 -- 172.21.15.92:0/2813103663 shutdown_connections 2024-03-19T01:52:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 1 -- 172.21.15.92:0/2813103663 wait complete. 2024-03-19T01:52:48.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.389+0000 7f3985423740 20 client.15273 trim_cache size 0 max 16384 2024-03-19T01:52:48.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:48 smithi144 ceph-mon[67199]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:48.875 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:48 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3622259959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f3985423740 1 Processor -- start 2024-03-19T01:52:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f3985423740 1 -- start start 2024-03-19T01:52:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f3985423740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f3985423740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650777499e0 0x56507775cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f3985423740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 0x565077755880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f3985423740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565077761c50 con 0x565077748ff0 2024-03-19T01:52:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f3985423740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565077762320 con 0x5650777499e0 2024-03-19T01:52:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f3985423740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650777629f0 con 0x56507774eb50 2024-03-19T01:52:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53180/0 (socket says 172.21.15.92:53180) 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397bfff640 1 -- 172.21.15.92:0/3203467654 learned_addr learned my addr 172.21.15.92:0/3203467654 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f3980cf8640 1 --2- 172.21.15.92:0/3203467654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 0x565077755880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397b7fe640 1 --2- 172.21.15.92:0/3203467654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650777499e0 0x56507775cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397bfff640 1 -- 172.21.15.92:0/3203467654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 msgr2=0x565077755880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397bfff640 1 --2- 172.21.15.92:0/3203467654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 0x565077755880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397bfff640 1 -- 172.21.15.92:0/3203467654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650777499e0 msgr2=0x56507775cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397bfff640 1 --2- 172.21.15.92:0/3203467654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650777499e0 0x56507775cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397bfff640 1 -- 172.21.15.92:0/3203467654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507774e130 con 0x565077748ff0 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.407+0000 7f397bfff640 1 --2- 172.21.15.92:0/3203467654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f396c003660 tx=0x7f396c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bbe44fc4286a3d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3961ffb640 1 -- 172.21.15.92:0/3203467654 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c0044e0 con 0x565077748ff0 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3961ffb640 1 -- 172.21.15.92:0/3203467654 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f396c01d5b0 con 0x565077748ff0 2024-03-19T01:52:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3961ffb640 1 -- 172.21.15.92:0/3203467654 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c031450 con 0x565077748ff0 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 -- 172.21.15.92:0/3203467654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 msgr2=0x565077755340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 --2- 172.21.15.92:0/3203467654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f396c003660 tx=0x7f396c003bd0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 -- 172.21.15.92:0/3203467654 shutdown_connections 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 --2- 172.21.15.92:0/3203467654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56507774eb50 0x565077755880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 --2- 172.21.15.92:0/3203467654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650777499e0 0x56507775cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 --2- 172.21.15.92:0/3203467654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565077748ff0 0x565077755340 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 -- 172.21.15.92:0/3203467654 >> 172.21.15.92:0/3203467654 conn(0x565077744560 msgr2=0x56507774aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 -- 172.21.15.92:0/3203467654 shutdown_connections 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 -- 172.21.15.92:0/3203467654 wait complete. 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.408+0000 7f3985423740 1 -- start start 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3985423740 1 Processor -- start 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3985423740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 0x565077758d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3985423740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 0x5650777592a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3985423740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56507774eb50 0x56507775c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3985423740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565077763790 con 0x56507774eb50 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3985423740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650777630c0 con 0x565077748ff0 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3985423740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565077763e60 con 0x5650777499e0 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3985423740 10 client.0 _ll_register_callbacks cb 0x7f396c005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3980cf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56507774eb50 0x56507775c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3980cf8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56507774eb50 0x56507775c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53184/0 (socket says 172.21.15.92:53184) 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3980cf8640 1 -- 172.21.15.92:0/710444668 learned_addr learned my addr 172.21.15.92:0/710444668 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f397b7fe640 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 0x5650777592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f397bfff640 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 0x565077758d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3980cf8640 1 -- 172.21.15.92:0/710444668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 msgr2=0x5650777592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3980cf8640 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 0x5650777592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3980cf8640 1 -- 172.21.15.92:0/710444668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 msgr2=0x565077758d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3980cf8640 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 0x565077758d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3980cf8640 1 -- 172.21.15.92:0/710444668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56507775c8b0 con 0x56507774eb50 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f3980cf8640 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56507774eb50 0x56507775c2c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f397409b210 tx=0x7f397409ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b6a3cae49d9c4d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.409+0000 7f395b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f395b7fe640 1 -- 172.21.15.92:0/710444668 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39740b3070 con 0x56507774eb50 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f395b7fe640 1 -- 172.21.15.92:0/710444668 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3974004540 con 0x56507774eb50 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f3985423740 1 -- 172.21.15.92:0/710444668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f396c01d5b0 con 0x56507774eb50 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f395b7fe640 1 -- 172.21.15.92:0/710444668 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39740a3a20 con 0x56507774eb50 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f3985423740 10 client.15279 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f3985423740 1 -- 172.21.15.92:0/710444668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650777c8d40 con 0x56507774eb50 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f395b7fe640 1 -- 172.21.15.92:0/710444668 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f396c01d5b0 con 0x56507774eb50 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f395b7fe640 1 -- 172.21.15.92:0/710444668 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f397400b040 con 0x56507774eb50 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f3985423740 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f3985423740 10 client.15279 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f3985423740 20 client.15279 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f3985423740 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x5650777cd9b0 0x5650777cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f3985423740 4 client.15279 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f3985423740 1 -- 172.21.15.92:0/710444668 --> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] -- command(tid 0: {"prefix": "status"}) -- 0x5650776b5670 con 0x5650777cd9b0 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.410+0000 7f397bfff640 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x5650777cd9b0 0x5650777cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f395b7fe640 10 client.15279 ms_handle_connect on v2:172.21.15.144:6838/380126062 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f397bfff640 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x5650777cd9b0 0x5650777cc6c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b54c844c8fd90a2 server_cookie=a14e2d2b21b8c214 in_seq=0 out_seq=0 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f395b7fe640 1 -- 172.21.15.92:0/710444668 <== mds.1 v2:172.21.15.144:6838/380126062 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5650776b5670 con 0x5650777cd9b0 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f395b7fe640 10 client.15279 handle_command_reply: tid=0 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 client.15279 shutdown 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 -- 172.21.15.92:0/710444668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56507774eb50 msgr2=0x56507775c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56507774eb50 0x56507775c2c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f397409b210 tx=0x7f397409ba40 comp rx=0 tx=0).stop 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 -- 172.21.15.92:0/710444668 shutdown_connections 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.144:6838/380126062,v1:172.21.15.144:6839/380126062] conn(0x5650777cd9b0 0x5650777cc6c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56507774eb50 0x56507775c2c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650777499e0 0x5650777592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f395b7fe640 0 client.15279 ms_handle_reset on v2:172.21.15.144:6838/380126062 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 --2- 172.21.15.92:0/710444668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565077748ff0 0x565077758d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 -- 172.21.15.92:0/710444668 >> 172.21.15.92:0/710444668 conn(0x565077744560 msgr2=0x565077755340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 -- 172.21.15.92:0/710444668 shutdown_connections 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 1 -- 172.21.15.92:0/710444668 wait complete. 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.411+0000 7f3985423740 20 client.15279 trim_cache size 0 max 16384 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:48.887 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 1, 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24476, 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 115.429126551, 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 124.134277662 2024-03-19T01:52:48.888 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:48.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 -- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39640a2800 msgr2=0x7f39640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39640a2800 0x7f39640a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f397c1017b0 tx=0x7f396c005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 -- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102310 msgr2=0x7f397c19e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102310 0x7f397c19e0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f397400efc0 tx=0x7f397400c800 comp rx=0 tx=0).stop 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 -- 172.21.15.92:0/2712846162 shutdown_connections 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102310 0x7f397c19e0d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f39640a2800 0x7f39640a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c101950 0x7f397c197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 --2- 172.21.15.92:0/2712846162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397c100760 0x7f397c196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 -- 172.21.15.92:0/2712846162 >> 172.21.15.92:0/2712846162 conn(0x7f397c0fbcd0 msgr2=0x7f397c104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 -- 172.21.15.92:0/2712846162 shutdown_connections 2024-03-19T01:52:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.412+0000 7f398275b640 1 -- 172.21.15.92:0/2712846162 wait complete. 2024-03-19T01:52:48.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-19T01:52:48.892 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:48 smithi092 ceph-mon[62574]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:48.892 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:48 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3622259959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:48.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:48 smithi159 ceph-mon[67110]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T01:52:48.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:48 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3622259959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T01:52:48.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.994+0000 7f41da59f640 1 -- 172.21.15.92:0/1505278225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4102220 msgr2=0x7f41d4104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.994+0000 7f41da59f640 1 --2- 172.21.15.92:0/1505278225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4102220 0x7f41d4104610 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009900 tx=0x7f41c402bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:48.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.994+0000 7f41da59f640 1 -- 172.21.15.92:0/1505278225 shutdown_connections 2024-03-19T01:52:48.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.994+0000 7f41da59f640 1 --2- 172.21.15.92:0/1505278225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d4107480 0x7f41d4109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.995 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.994+0000 7f41da59f640 1 --2- 172.21.15.92:0/1505278225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d4104b50 0x7f41d4106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.994+0000 7f41da59f640 1 --2- 172.21.15.92:0/1505278225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4102220 0x7f41d4104610 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.994+0000 7f41da59f640 1 -- 172.21.15.92:0/1505278225 >> 172.21.15.92:0/1505278225 conn(0x7f41d40fbb90 msgr2=0x7f41d40fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:48.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 -- 172.21.15.92:0/1505278225 shutdown_connections 2024-03-19T01:52:48.996 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 -- 172.21.15.92:0/1505278225 wait complete. 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 Processor -- start 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 -- start start 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d4102220 0x7f41d4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4104b50 0x7f41d4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d4107480 0x7f41d419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f41d410d390 con 0x7f41d4104b50 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41d410d210 con 0x7f41d4102220 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41da59f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41d410d510 con 0x7f41d4107480 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41d8d9c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4104b50 0x7f41d4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.998 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.995+0000 7f41d8d9c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4104b50 0x7f41d4197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53202/0 (socket says 172.21.15.92:53202) 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41d8d9c640 1 -- 172.21.15.92:0/1322459544 learned_addr learned my addr 172.21.15.92:0/1322459544 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41d9d9e640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d4107480 0x7f41d419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41d959d640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d4102220 0x7f41d4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41d8d9c640 1 -- 172.21.15.92:0/1322459544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d4107480 msgr2=0x7f41d419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d4107480 0x7f41d419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41d8d9c640 1 -- 172.21.15.92:0/1322459544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d4102220 msgr2=0x7f41d4197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d4102220 0x7f41d4197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41d8d9c640 1 -- 172.21.15.92:0/1322459544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d419f620 con 0x7f41d4104b50 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4104b50 0x7f41d4197f40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f41d0005fd0 tx=0x7f41d000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41ca7fc640 1 -- 172.21.15.92:0/1322459544 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0017070 con 0x7f41d4104b50 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41da59f640 1 -- 172.21.15.92:0/1322459544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d419f910 con 0x7f41d4104b50 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41ca7fc640 1 -- 172.21.15.92:0/1322459544 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d0009c30 con 0x7f41d4104b50 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41ca7fc640 1 -- 172.21.15.92:0/1322459544 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0010460 con 0x7f41d4104b50 2024-03-19T01:52:48.999 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.996+0000 7f41da59f640 1 -- 172.21.15.92:0/1322459544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d419fe60 con 0x7f41d4104b50 2024-03-19T01:52:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.998+0000 7f41ca7fc640 1 -- 172.21.15.92:0/1322459544 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f41d0003e70 con 0x7f41d4104b50 2024-03-19T01:52:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41de3e0740 1 Processor -- start 2024-03-19T01:52:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41de3e0740 1 -- start start 2024-03-19T01:52:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4ed5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e4f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41de3e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555d0e451de0 con 0x555d0e4ed1c0 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41de3e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d0e3fac80 con 0x555d0e4f2d20 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41de3e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d0e44de80 con 0x555d0e4edbb0 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41ca7fc640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41c00a2800 0x7f41c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d9d9e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4ed5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d959d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e4f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d9d9e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4ed5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53216/0 (socket says 172.21.15.92:53216) 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d9d9e640 1 -- 172.21.15.92:0/4196394978 learned_addr learned my addr 172.21.15.92:0/4196394978 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d9d9e640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41c00a2800 0x7f41c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41ca7fc640 1 -- 172.21.15.92:0/1322459544 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f41d00ce700 con 0x7f41d4104b50 2024-03-19T01:52:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d959d640 1 -- 172.21.15.92:0/4196394978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 msgr2=0x555d0e4f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/4196394978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d959d640 1 --2- 172.21.15.92:0/4196394978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d959d640 1 -- 172.21.15.92:0/4196394978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 msgr2=0x555d0e4ed5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d959d640 1 --2- 172.21.15.92:0/4196394978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4ed5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d959d640 1 -- 172.21.15.92:0/4196394978 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0e40cfb0 con 0x555d0e4edbb0 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:48.999+0000 7f41d9d9e640 1 --2- 172.21.15.92:0/4196394978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4ed5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.000+0000 7f41d9d9e640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41c00a2800 0x7f41c00a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x555d0e4ee410 tx=0x7f41cc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.000+0000 7f41d959d640 1 --2- 172.21.15.92:0/4196394978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e4f26b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009230 tx=0x7f41c40095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0da8a5fd3fd904a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.000+0000 7f41bf7fe640 1 -- 172.21.15.92:0/4196394978 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c402cb80 con 0x555d0e4edbb0 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.000+0000 7f41bf7fe640 1 -- 172.21.15.92:0/4196394978 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c4005040 con 0x555d0e4edbb0 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.000+0000 7f41bf7fe640 1 -- 172.21.15.92:0/4196394978 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4004920 con 0x555d0e4edbb0 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 -- 172.21.15.92:0/4196394978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 msgr2=0x555d0e4f26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 --2- 172.21.15.92:0/4196394978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e4f26b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009230 tx=0x7f41c40095e0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 -- 172.21.15.92:0/4196394978 shutdown_connections 2024-03-19T01:52:49.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 --2- 172.21.15.92:0/4196394978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 --2- 172.21.15.92:0/4196394978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e4f26b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 --2- 172.21.15.92:0/4196394978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4ed5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 -- 172.21.15.92:0/4196394978 >> 172.21.15.92:0/4196394978 conn(0x555d0e4e8730 msgr2=0x555d0e4e8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 -- 172.21.15.92:0/4196394978 shutdown_connections 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 -- 172.21.15.92:0/4196394978 wait complete. 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 -- start start 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 Processor -- start 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4fec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555d0e4f5c70 con 0x555d0e4edbb0 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d0e40dfc0 con 0x555d0e4f2d20 2024-03-19T01:52:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.001+0000 7f41de3e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d0e4f5df0 con 0x555d0e4ed1c0 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41de3e0740 10 client.0 _ll_register_callbacks cb 0x555d0e449a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d8d9c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d8d9c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53228/0 (socket says 172.21.15.92:53228) 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d8d9c640 1 -- 172.21.15.92:0/257852915 learned_addr learned my addr 172.21.15.92:0/257852915 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d959d640 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d9d9e640 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4fec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d8d9c640 1 -- 172.21.15.92:0/257852915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4ed1c0 msgr2=0x555d0e4f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d8d9c640 1 -- 172.21.15.92:0/257852915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 msgr2=0x555d0e4fec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4fec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d8d9c640 1 -- 172.21.15.92:0/257852915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0e1662d0 con 0x555d0e4edbb0 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f41d00a3320 tx=0x7f41d009af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e4205f9a83fc4d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41bcff9640 1 -- 172.21.15.92:0/257852915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d009b0b0 con 0x555d0e4edbb0 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41bcff9640 1 -- 172.21.15.92:0/257852915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f41d00b9bb0 con 0x555d0e4edbb0 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41de3e0740 1 -- 172.21.15.92:0/257852915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c4005040 con 0x555d0e4edbb0 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41bcff9640 1 -- 172.21.15.92:0/257852915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f41d009f200 con 0x555d0e4edbb0 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41de3e0740 10 client.15291 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.002+0000 7f41de3e0740 1 -- 172.21.15.92:0/257852915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d0e2306b0 con 0x555d0e4edbb0 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41bcff9640 1 -- 172.21.15.92:0/257852915 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c4005040 con 0x555d0e4edbb0 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41bcff9640 1 -- 172.21.15.92:0/257852915 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f41d009a500 con 0x555d0e4edbb0 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41de3e0740 10 client.15291 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41de3e0740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41de3e0740 20 client.15291 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41de3e0740 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x555d0e504480 0x555d0e5058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41de3e0740 4 client.15291 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41de3e0740 1 -- 172.21.15.92:0/257852915 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d0e45f320 con 0x555d0e504480 2024-03-19T01:52:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41d959d640 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x555d0e504480 0x555d0e5058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41bcff9640 10 client.15291 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:52:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.003+0000 7f41d959d640 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x555d0e504480 0x555d0e5058f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c822442962cc4e46 server_cookie=796d7f42a1c7d72b in_seq=0 out_seq=0 2024-03-19T01:52:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.004+0000 7f41bcff9640 1 -- 172.21.15.92:0/257852915 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555d0e45f320 con 0x555d0e504480 2024-03-19T01:52:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.004+0000 7f41bcff9640 10 client.15291 handle_command_reply: tid=0 2024-03-19T01:52:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.004+0000 7f41de3e0740 1 client.15291 shutdown 2024-03-19T01:52:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.004+0000 7f41de3e0740 1 -- 172.21.15.92:0/257852915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4edbb0 msgr2=0x555d0e4f7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.004+0000 7f41de3e0740 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f41d00a3320 tx=0x7f41d009af90 comp rx=0 tx=0).stop 2024-03-19T01:52:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41de3e0740 1 -- 172.21.15.92:0/257852915 shutdown_connections 2024-03-19T01:52:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41de3e0740 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x555d0e504480 0x555d0e5058f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41de3e0740 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4fec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41bcff9640 0 client.15291 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:52:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41de3e0740 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41de3e0740 1 --2- 172.21.15.92:0/257852915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41de3e0740 1 -- 172.21.15.92:0/257852915 >> 172.21.15.92:0/257852915 conn(0x555d0e4e8730 msgr2=0x555d0e4e79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41de3e0740 1 -- 172.21.15.92:0/257852915 shutdown_connections 2024-03-19T01:52:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41de3e0740 1 -- 172.21.15.92:0/257852915 wait complete. 2024-03-19T01:52:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.005+0000 7f41de3e0740 20 client.15291 trim_cache size 0 max 16384 2024-03-19T01:52:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41de3e0740 1 Processor -- start 2024-03-19T01:52:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41de3e0740 1 -- start start 2024-03-19T01:52:49.038 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4febd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41de3e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555d0e503c30 con 0x555d0e4ed1c0 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41de3e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d0e504300 con 0x555d0e4f2d20 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41de3e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d0e5049d0 con 0x555d0e4edbb0 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d959d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d959d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53232/0 (socket says 172.21.15.92:53232) 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d959d640 1 -- 172.21.15.92:0/776208599 learned_addr learned my addr 172.21.15.92:0/776208599 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/776208599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d9d9e640 1 --2- 172.21.15.92:0/776208599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d959d640 1 -- 172.21.15.92:0/776208599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 msgr2=0x555d0e4f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d959d640 1 --2- 172.21.15.92:0/776208599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.039 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d959d640 1 -- 172.21.15.92:0/776208599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 msgr2=0x555d0e4febd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d959d640 1 --2- 172.21.15.92:0/776208599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4febd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.032+0000 7f41d959d640 1 -- 172.21.15.92:0/776208599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0e4e3880 con 0x555d0e4ed1c0 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41d959d640 1 --2- 172.21.15.92:0/776208599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009900 tx=0x7f41c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3be13e59923ec95 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41beffd640 1 -- 172.21.15.92:0/776208599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4004230 con 0x555d0e4ed1c0 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41beffd640 1 -- 172.21.15.92:0/776208599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c402cbc0 con 0x555d0e4ed1c0 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41beffd640 1 -- 172.21.15.92:0/776208599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4004bd0 con 0x555d0e4ed1c0 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41de3e0740 1 -- 172.21.15.92:0/776208599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 msgr2=0x555d0e4f7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41de3e0740 1 --2- 172.21.15.92:0/776208599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009900 tx=0x7f41c4005fd0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41de3e0740 1 -- 172.21.15.92:0/776208599 shutdown_connections 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41de3e0740 1 --2- 172.21.15.92:0/776208599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e4febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41de3e0740 1 --2- 172.21.15.92:0/776208599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e4f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41de3e0740 1 --2- 172.21.15.92:0/776208599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e4f7690 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.040 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41de3e0740 1 -- 172.21.15.92:0/776208599 >> 172.21.15.92:0/776208599 conn(0x555d0e4e8730 msgr2=0x555d0e4e8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41de3e0740 1 -- 172.21.15.92:0/776208599 shutdown_connections 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.033+0000 7f41de3e0740 1 -- 172.21.15.92:0/776208599 wait complete. 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41de3e0740 1 -- start start 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41de3e0740 1 Processor -- start 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e5681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e56a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41de3e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e56cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41de3e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555d0e505770 con 0x555d0e4ed1c0 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41de3e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d0e5050a0 con 0x555d0e4f2d20 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41de3e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d0e505e40 con 0x555d0e4edbb0 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41de3e0740 10 client.0 _ll_register_callbacks cb 0x7f41d000df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41d959d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e5681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41d959d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e5681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53246/0 (socket says 172.21.15.92:53246) 2024-03-19T01:52:49.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41d959d640 1 -- 172.21.15.92:0/582256652 learned_addr learned my addr 172.21.15.92:0/582256652 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e56a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41d9d9e640 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e56cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41d959d640 1 -- 172.21.15.92:0/582256652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 msgr2=0x555d0e56a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41d959d640 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e56a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41d959d640 1 -- 172.21.15.92:0/582256652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 msgr2=0x555d0e56cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.034+0000 7f41d959d640 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e56cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41d959d640 1 -- 172.21.15.92:0/582256652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0e56d290 con 0x555d0e4ed1c0 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41d9d9e640 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e56cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41d959d640 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e5681e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f41c402cfd0 tx=0x7f41c40095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d3f63dde4821b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41bcff9640 1 -- 172.21.15.92:0/582256652 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4004030 con 0x555d0e4ed1c0 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41de3e0740 1 -- 172.21.15.92:0/582256652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c402cbc0 con 0x555d0e4ed1c0 2024-03-19T01:52:49.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41bcff9640 1 -- 172.21.15.92:0/582256652 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f41c40041d0 con 0x555d0e4ed1c0 2024-03-19T01:52:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41bcff9640 1 -- 172.21.15.92:0/582256652 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f41c40043d0 con 0x555d0e4ed1c0 2024-03-19T01:52:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41de3e0740 10 client.15297 fetch_fsmap learned FSMap version 26 2024-03-19T01:52:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41de3e0740 1 -- 172.21.15.92:0/582256652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d0e56e590 con 0x555d0e4ed1c0 2024-03-19T01:52:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41bcff9640 1 -- 172.21.15.92:0/582256652 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c402cbc0 con 0x555d0e4ed1c0 2024-03-19T01:52:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41bcff9640 1 -- 172.21.15.92:0/582256652 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f41c4050030 con 0x555d0e4ed1c0 2024-03-19T01:52:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41de3e0740 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41de3e0740 10 client.15297 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41de3e0740 20 client.15297 populate_metadata read hostname 'smithi092' 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41de3e0740 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x555d0e572b10 0x555d0e571820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41de3e0740 4 client.15297 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.035+0000 7f41de3e0740 1 -- 172.21.15.92:0/582256652 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "status"}) -- 0x555d0e459670 con 0x555d0e572b10 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x555d0e572b10 0x555d0e571820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41bcff9640 10 client.15297 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41d8d9c640 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x555d0e572b10 0x555d0e571820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96f9e5c3850ee3c server_cookie=e5c271dbea7c3b68 in_seq=0 out_seq=0 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41bcff9640 1 -- 172.21.15.92:0/582256652 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555d0e459670 con 0x555d0e572b10 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41bcff9640 10 client.15297 handle_command_reply: tid=0 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 client.15297 shutdown 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 -- 172.21.15.92:0/582256652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 msgr2=0x555d0e5681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e5681e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f41c402cfd0 tx=0x7f41c40095e0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 -- 172.21.15.92:0/582256652 shutdown_connections 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x555d0e572b10 0x555d0e571820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d0e4f2d20 0x555d0e56cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d0e4edbb0 0x555d0e56a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41bcff9640 0 client.15297 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 --2- 172.21.15.92:0/582256652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d0e4ed1c0 0x555d0e5681e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 -- 172.21.15.92:0/582256652 >> 172.21.15.92:0/582256652 conn(0x555d0e4e8730 msgr2=0x555d0e4f6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 -- 172.21.15.92:0/582256652 shutdown_connections 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.036+0000 7f41de3e0740 1 -- 172.21.15.92:0/582256652 wait complete. 2024-03-19T01:52:49.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41de3e0740 20 client.15297 trim_cache size 0 max 16384 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "d1e857f4-e591-11ee-95c9-87774f69a715", 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 24493, 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 118.50401031600001, 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 26, 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 87, 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 85, 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 133.46319317000001 2024-03-19T01:52:49.046 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T01:52:49.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 -- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41c00a2800 msgr2=0x7f41c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:49.047 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41c00a2800 0x7f41c00a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x555d0e4ee410 tx=0x7f41cc009500 comp rx=0 tx=0).stop 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 -- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4104b50 msgr2=0x7f41d4197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4104b50 0x7f41d4197f40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f41d0005fd0 tx=0x7f41d000dde0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 -- 172.21.15.92:0/1322459544 shutdown_connections 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f41c00a2800 0x7f41c00a4cc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d4107480 0x7f41d419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41d4104b50 0x7f41d4197f40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 --2- 172.21.15.92:0/1322459544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d4102220 0x7f41d4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 -- 172.21.15.92:0/1322459544 >> 172.21.15.92:0/1322459544 conn(0x7f41d40fbb90 msgr2=0x7f41d4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 -- 172.21.15.92:0/1322459544 shutdown_connections 2024-03-19T01:52:49.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:49.037+0000 7f41da59f640 1 -- 172.21.15.92:0/1322459544 wait complete. 2024-03-19T01:52:49.049 INFO:teuthology.run_tasks:Running task kclient... 2024-03-19T01:52:49.096 INFO:tasks.kclient:Mounting kernel clients... 2024-03-19T01:52:49.096 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-19T01:52:49.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:52:49.098 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-03-19T01:52:49.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:52:49.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-03-19T01:52:49.186 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2024-03-19T01:52:49.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T01:52:49.188 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-19T01:52:49.188 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-19T01:52:49.188 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-19T01:52:49.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-19T01:52:49.473 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T01:52:50.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:50 smithi144 ceph-mon[67199]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:50.955 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:50 smithi092 ceph-mon[62574]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:50 smithi159 ceph-mon[67110]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.328+0000 7f0933daf700 1 -- 172.21.15.92:0/1204565693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c107690 msgr2=0x7f092c10f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.328+0000 7f0933daf700 1 --2- 172.21.15.92:0/1204565693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c107690 0x7f092c10f210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f092400b0d0 tx=0x7f092400b3e0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.328+0000 7f0933daf700 1 -- 172.21.15.92:0/1204565693 shutdown_connections 2024-03-19T01:52:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.328+0000 7f0933daf700 1 --2- 172.21.15.92:0/1204565693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c107690 0x7f092c10f210 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.328+0000 7f0933daf700 1 --2- 172.21.15.92:0/1204565693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092c06aab0 0x7f092c06ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.328+0000 7f0933daf700 1 --2- 172.21.15.92:0/1204565693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f092c06b460 0x7f092c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.328+0000 7f0933daf700 1 -- 172.21.15.92:0/1204565693 >> 172.21.15.92:0/1204565693 conn(0x7f092c077260 msgr2=0x7f092c077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.328+0000 7f0933daf700 1 -- 172.21.15.92:0/1204565693 shutdown_connections 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.328+0000 7f0933daf700 1 -- 172.21.15.92:0/1204565693 wait complete. 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f0933daf700 1 Processor -- start 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f0933daf700 1 -- start start 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f0933daf700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f092c06aab0 0x7f092c101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f0933daf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c06b460 0x7f092c101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f0933daf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092c107690 0x7f092c101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f0933daf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f092c112650 con 0x7f092c06b460 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f0933daf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f092c1124d0 con 0x7f092c107690 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f0933daf700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f092c1127d0 con 0x7f092c06aab0 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093134a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c06b460 0x7f092c101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093134a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c06b460 0x7f092c101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53276/0 (socket says 172.21.15.92:53276) 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093134a700 1 -- 172.21.15.92:0/2887252509 learned_addr learned my addr 172.21.15.92:0/2887252509 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f0931b4b700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f092c06aab0 0x7f092c101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093234c700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092c107690 0x7f092c101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093134a700 1 -- 172.21.15.92:0/2887252509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f092c06aab0 msgr2=0x7f092c101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093134a700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f092c06aab0 0x7f092c101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093134a700 1 -- 172.21.15.92:0/2887252509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092c107690 msgr2=0x7f092c101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093134a700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092c107690 0x7f092c101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093134a700 1 -- 172.21.15.92:0/2887252509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c1a7070 con 0x7f092c06b460 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.329+0000 7f093134a700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c06b460 0x7f092c101850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f091c00e9c0 tx=0x7f091c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.330+0000 7f0922ffd700 1 -- 172.21.15.92:0/2887252509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c004a30 con 0x7f092c06b460 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.330+0000 7f0922ffd700 1 -- 172.21.15.92:0/2887252509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f091c00cca0 con 0x7f092c06b460 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.330+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092c1a7360 con 0x7f092c06b460 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.330+0000 7f0922ffd700 1 -- 172.21.15.92:0/2887252509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c005690 con 0x7f092c06b460 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.330+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092c1a7850 con 0x7f092c06b460 2024-03-19T01:52:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.330+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092c068fd0 con 0x7f092c06b460 2024-03-19T01:52:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.332+0000 7f0922ffd700 1 -- 172.21.15.92:0/2887252509 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f091c004cf0 con 0x7f092c06b460 2024-03-19T01:52:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.332+0000 7f0922ffd700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f09180a2890 0x7f09180a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.332+0000 7f0931b4b700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f09180a2890 0x7f09180a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.333+0000 7f0922ffd700 1 -- 172.21.15.92:0/2887252509 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f091c0cebb0 con 0x7f092c06b460 2024-03-19T01:52:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.333+0000 7f0931b4b700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f09180a2890 0x7f09180a4d50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0928009c60 tx=0x7f0928009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:51.340 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.337+0000 7f0922ffd700 1 -- 172.21.15.92:0/2887252509 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f091c09bc60 con 0x7f092c06b460 2024-03-19T01:52:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.584+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f092c02dbb0 con 0x7f092c06b460 2024-03-19T01:52:51.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.586+0000 7f0922ffd700 1 -- 172.21.15.92:0/2887252509 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f091c09b560 con 0x7f092c06b460 2024-03-19T01:52:51.587 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-19T01:52:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f09180a2890 msgr2=0x7f09180a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f09180a2890 0x7f09180a4d50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0928009c60 tx=0x7f0928009260 comp rx=0 tx=0).stop 2024-03-19T01:52:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c06b460 msgr2=0x7f092c101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c06b460 0x7f092c101850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f091c00e9c0 tx=0x7f091c00ecd0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 shutdown_connections 2024-03-19T01:52:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f09180a2890 0x7f09180a4d50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092c107690 0x7f092c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f092c06aab0 0x7f092c101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 --2- 172.21.15.92:0/2887252509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f092c06b460 0x7f092c101850 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:51.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.588+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 >> 172.21.15.92:0/2887252509 conn(0x7f092c077260 msgr2=0x7f092c0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:51.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.589+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 shutdown_connections 2024-03-19T01:52:51.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:51.589+0000 7f0933daf700 1 -- 172.21.15.92:0/2887252509 wait complete. 2024-03-19T01:52:52.351 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-19T01:52:52.351 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-19T01:52:52.351 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2024-03-19T01:52:52.352 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-19T01:52:52.352 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-19T01:52:52.352 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-19T01:52:52.352 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-19T01:52:52.352 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-19T01:52:52.352 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-19T01:52:52.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:52:52.352 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2024-03-19T01:52:52.369 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-19T01:52:52.369 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-19T01:52:52.369 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-03-19T01:52:52.369 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-03-19T01:52:52.369 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-03-19T01:52:52.369 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-03-19T01:52:52.369 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T01:52:52.369 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:52:52.369 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 40658sec preferred_lft 40658sec 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2024-03-19T01:52:52.370 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-03-19T01:52:52.371 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-19T01:52:52.371 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:52:52.371 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-03-19T01:52:52.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2024-03-19T01:52:52.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2024-03-19T01:52:52.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2024-03-19T01:52:52.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-19T01:52:52.371 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-03-19T01:52:52.458 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:52:52.569 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:52:52.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:52:52.576 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-19T01:52:52.604 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-03-19T01:52:52.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:52:52.606 DEBUG:teuthology.orchestra.run.smithi092:> route 2024-03-19T01:52:52.674 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2024-03-19T01:52:52.674 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-19T01:52:52.674 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-19T01:52:52.675 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-19T01:52:52.675 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-19T01:52:52.675 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:52:52.675 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-03-19T01:52:52.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-19T01:52:52.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-19T01:52:52.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-19T01:52:52.675 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-03-19T01:52:52.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:52:52.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:52 smithi144 ceph-mon[67199]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:52.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:52 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2887252509' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T01:52:52.866 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:52 smithi092 ceph-mon[62574]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:52.866 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:52 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2887252509' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T01:52:52.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:52:52.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:52:52.954 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-03-19T01:52:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:52 smithi159 ceph-mon[67110]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T01:52:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:52 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2887252509' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-19T01:52:52.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:52:52.971 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2024-03-19T01:52:53.024 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:52:53.024 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-03-19T01:52:53.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T01:52:53.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-19T01:52:53.024 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-03-19T01:52:53.112 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:52:53.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:52:53.147 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-19T01:52:53.148 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:52:53.148 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-03-19T01:52:53.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-19T01:52:53.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-19T01:52:53.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-19T01:52:53.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-19T01:52:53.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-19T01:52:53.148 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-03-19T01:52:53.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:52:53.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:52:53.281 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-19T01:52:53.281 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-03-19T01:52:53.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2024-03-19T01:52:53.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2024-03-19T01:52:53.281 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-03-19T01:52:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:52:53.358 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:52:53.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T01:52:53.365 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-19T01:52:53.380 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-19T01:52:53.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-19T01:52:53.381 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-19T01:52:53.436 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-19T01:52:53.436 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-19T01:52:53.436 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/9e966a37-f1ed-4f65-93aa-7bf90d627330 2024-03-19T01:52:53.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:52:53.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/9e966a37-f1ed-4f65-93aa-7bf90d627330 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-19T01:52:53.520 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-19T01:52:53.520 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-03-19T01:52:53.520 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2024-03-19T01:52:53.667 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-03-19T01:52:53.667 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-03-19T01:52:53.667 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2024-03-19T01:52:53.667 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.144:3300,172.21.15.159:3300" 2024-03-19T01:52:53.667 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:3300,172.21.15.144:3300,172.21.15.159:3300:/volumes/qa/sv_1/9e966a37-f1ed-4f65-93aa-7bf90d627330 2024-03-19T01:52:53.667 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=d1e857f4-e591-11ee-95c9-87774f69a715" will pass to kernel 2024-03-19T01:52:53.668 INFO:teuthology.orchestra.run.smithi092.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-19T01:52:53.668 INFO:teuthology.orchestra.run.smithi092.stdout: You just mounted a file system that supports labels which does not 2024-03-19T01:52:53.668 INFO:teuthology.orchestra.run.smithi092.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-19T01:52:53.668 INFO:teuthology.orchestra.run.smithi092.stdout: applications will generate AVC messages and not be allowed access to 2024-03-19T01:52:53.668 INFO:teuthology.orchestra.run.smithi092.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-19T01:52:53.670 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-19T01:52:53.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T01:52:53.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-19T01:52:53.741 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-19T01:52:53.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-19T01:52:53.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:52:53.839 INFO:teuthology.orchestra.run.smithi092.stdout:client15306 2024-03-19T01:52:53.839 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:52:53.840 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T01:52:53.856 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-19T01:52:53.945 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-19T01:52:53.947 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:52:53.953 INFO:teuthology.orchestra.run.smithi092.stdout:client15306 2024-03-19T01:52:53.953 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-19T01:52:54.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-19T01:52:54.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:52:54.009 INFO:teuthology.orchestra.run.smithi092.stdout:client15306 2024-03-19T01:52:54.009 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d1e857f4-e591-11ee-95c9-87774f69a715.client15306/status') 2024-03-19T01:52:54.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-19T01:52:54.071 INFO:teuthology.orchestra.run.smithi092.stdout:instance: client.15306 (3)192.168.0.1:0/224320916 2024-03-19T01:52:54.071 INFO:teuthology.orchestra.run.smithi092.stdout:blocklisted: false 2024-03-19T01:52:54.072 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2024-03-19T01:52:54.072 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2024-03-19T01:52:54.072 INFO:teuthology.orchestra.run.smithi092.stderr:69 bytes copied, 2.1151e-05 s, 3.3 MB/s 2024-03-19T01:52:54.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-19T01:52:54.078 INFO:teuthology.run_tasks:Running task exec... 2024-03-19T01:52:54.089 INFO:teuthology.task.exec:Executing custom commands... 2024-03-19T01:52:54.089 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-03-19T01:52:54.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-19T01:52:54.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 -- 172.21.15.92:0/4084979713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 msgr2=0x7ff73c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 --2- 172.21.15.92:0/4084979713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 0x7ff73c101db0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff730009900 tx=0x7ff73002bc30 comp rx=0 tx=0).stop 2024-03-19T01:52:54.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 -- 172.21.15.92:0/4084979713 shutdown_connections 2024-03-19T01:52:54.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 --2- 172.21.15.92:0/4084979713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff73c1022f0 0x7ff73c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 --2- 172.21.15.92:0/4084979713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 0x7ff73c101db0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 --2- 172.21.15.92:0/4084979713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c100740 0x7ff73c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 -- 172.21.15.92:0/4084979713 >> 172.21.15.92:0/4084979713 conn(0x7ff73c0fbcb0 msgr2=0x7ff73c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 -- 172.21.15.92:0/4084979713 shutdown_connections 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 -- 172.21.15.92:0/4084979713 wait complete. 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 Processor -- start 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.195+0000 7ff74184b640 1 -- start start 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff74184b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c100740 0x7ff73c195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff74184b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 0x7ff73c195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff74184b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff73c1022f0 0x7ff73c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff74184b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c10b2c0 con 0x7ff73c101930 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff74184b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c10b140 con 0x7ff73c100740 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff74184b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c10b440 con 0x7ff73c1022f0 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 0x7ff73c195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 0x7ff73c195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48038/0 (socket says 172.21.15.92:48038) 2024-03-19T01:52:54.198 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73a7fc640 1 -- 172.21.15.92:0/3111052990 learned_addr learned my addr 172.21.15.92:0/3111052990 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73affd640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c100740 0x7ff73c195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73b7fe640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff73c1022f0 0x7ff73c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73a7fc640 1 -- 172.21.15.92:0/3111052990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff73c1022f0 msgr2=0x7ff73c19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73a7fc640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff73c1022f0 0x7ff73c19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73a7fc640 1 -- 172.21.15.92:0/3111052990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c100740 msgr2=0x7ff73c195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73a7fc640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c100740 0x7ff73c195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73a7fc640 1 -- 172.21.15.92:0/3111052990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c19d4b0 con 0x7ff73c101930 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff73a7fc640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 0x7ff73c195dd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff730009230 tx=0x7ff73002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff740849640 1 -- 172.21.15.92:0/3111052990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730041070 con 0x7ff73c101930 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c19d740 con 0x7ff73c101930 2024-03-19T01:52:54.199 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff740849640 1 -- 172.21.15.92:0/3111052990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff730004450 con 0x7ff73c101930 2024-03-19T01:52:54.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff740849640 1 -- 172.21.15.92:0/3111052990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730003e20 con 0x7ff73c101930 2024-03-19T01:52:54.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.196+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c19dc80 con 0x7ff73c101930 2024-03-19T01:52:54.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.197+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c107a00 con 0x7ff73c101930 2024-03-19T01:52:54.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.199+0000 7ff740849640 1 -- 172.21.15.92:0/3111052990 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff730044070 con 0x7ff73c101930 2024-03-19T01:52:54.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.200+0000 7ff740849640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7240a28a0 0x7ff7240a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:54.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.200+0000 7ff73affd640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7240a28a0 0x7ff7240a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:54.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.200+0000 7ff740849640 1 -- 172.21.15.92:0/3111052990 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff7300f0950 con 0x7ff73c101930 2024-03-19T01:52:54.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.200+0000 7ff73affd640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7240a28a0 0x7ff7240a4d60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff73c101790 tx=0x7ff72c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:54.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.203+0000 7ff740849640 1 -- 172.21.15.92:0/3111052990 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff7300bd3b0 con 0x7ff73c101930 2024-03-19T01:52:54.424 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.425+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7ff73c02d1f0 con 0x7ff73c101930 2024-03-19T01:52:54.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.588+0000 7ff740849640 1 -- 172.21.15.92:0/3111052990 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7ff7300c1450 con 0x7ff73c101930 2024-03-19T01:52:54.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7240a28a0 msgr2=0x7ff7240a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7240a28a0 0x7ff7240a4d60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff73c101790 tx=0x7ff72c00d040 comp rx=0 tx=0).stop 2024-03-19T01:52:54.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 msgr2=0x7ff73c195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 0x7ff73c195dd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff730009230 tx=0x7ff73002c650 comp rx=0 tx=0).stop 2024-03-19T01:52:54.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 shutdown_connections 2024-03-19T01:52:54.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff7240a28a0 0x7ff7240a4d60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff73c1022f0 0x7ff73c19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff73c101930 0x7ff73c195dd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 --2- 172.21.15.92:0/3111052990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c100740 0x7ff73c195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.590+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 >> 172.21.15.92:0/3111052990 conn(0x7ff73c0fbcb0 msgr2=0x7ff73c104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:54.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.591+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 shutdown_connections 2024-03-19T01:52:54.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.591+0000 7ff74184b640 1 -- 172.21.15.92:0/3111052990 wait complete. 2024-03-19T01:52:54.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-19T01:52:54.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 -- 172.21.15.92:0/2112784104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 msgr2=0x7fe108101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 --2- 172.21.15.92:0/2112784104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 0x7fe108101d80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009860 tx=0x7fe0fc02bb70 comp rx=0 tx=0).stop 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 -- 172.21.15.92:0/2112784104 shutdown_connections 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 --2- 172.21.15.92:0/2112784104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1081022c0 0x7fe108108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 --2- 172.21.15.92:0/2112784104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 0x7fe108101d80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 --2- 172.21.15.92:0/2112784104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108100710 0x7fe108100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 -- 172.21.15.92:0/2112784104 >> 172.21.15.92:0/2112784104 conn(0x7fe1080fbc80 msgr2=0x7fe1080fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 -- 172.21.15.92:0/2112784104 shutdown_connections 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 -- 172.21.15.92:0/2112784104 wait complete. 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 Processor -- start 2024-03-19T01:52:54.712 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.712+0000 7fe10e777640 1 -- start start 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe10e777640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108100710 0x7fe108069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe10e777640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 0x7fe10819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe10e777640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1081022c0 0x7fe10819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe10e777640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe10810c390 con 0x7fe108101900 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe10e777640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe10810c210 con 0x7fe108100710 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe10e777640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe10810c510 con 0x7fe1081022c0 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe1077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 0x7fe10819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe1077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 0x7fe10819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48060/0 (socket says 172.21.15.92:48060) 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe1077fe640 1 -- 172.21.15.92:0/783076958 learned_addr learned my addr 172.21.15.92:0/783076958 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe107fff640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108100710 0x7fe108069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:54.714 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe10cd14640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1081022c0 0x7fe10819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe1077fe640 1 -- 172.21.15.92:0/783076958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1081022c0 msgr2=0x7fe10819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe1077fe640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1081022c0 0x7fe10819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe1077fe640 1 -- 172.21.15.92:0/783076958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108100710 msgr2=0x7fe108069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe1077fe640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108100710 0x7fe108069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe1077fe640 1 -- 172.21.15.92:0/783076958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10819f720 con 0x7fe108101900 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.713+0000 7fe1077fe640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 0x7fe10819c7f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005940 tx=0x7fe0fc032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.714+0000 7fe1057fa640 1 -- 172.21.15.92:0/783076958 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc041020 con 0x7fe108101900 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.714+0000 7fe1057fa640 1 -- 172.21.15.92:0/783076958 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0fc02c9f0 con 0x7fe108101900 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.714+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10819f950 con 0x7fe108101900 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.714+0000 7fe1057fa640 1 -- 172.21.15.92:0/783076958 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc039040 con 0x7fe108101900 2024-03-19T01:52:54.715 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.714+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10819fe40 con 0x7fe108101900 2024-03-19T01:52:54.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.715+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe108107a30 con 0x7fe108101900 2024-03-19T01:52:54.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.716+0000 7fe1057fa640 1 -- 172.21.15.92:0/783076958 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe0fc044020 con 0x7fe108101900 2024-03-19T01:52:54.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.716+0000 7fe1057fa640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0f00a28a0 0x7fe0f00a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:52:54.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.717+0000 7fe107fff640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0f00a28a0 0x7fe0f00a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:52:54.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.717+0000 7fe1057fa640 1 -- 172.21.15.92:0/783076958 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe0fc0f03a0 con 0x7fe108101900 2024-03-19T01:52:54.716 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.717+0000 7fe107fff640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0f00a28a0 0x7fe0f00a4d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe108101760 tx=0x7fe0f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:52:54.720 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.721+0000 7fe1057fa640 1 -- 172.21.15.92:0/783076958 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe0fc04b020 con 0x7fe108101900 2024-03-19T01:52:54.840 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:54 smithi092 ceph-mon[62574]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:54.841 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:54 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3111052990' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T01:52:54.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:54 smithi144 ceph-mon[67199]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:54.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:54 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3111052990' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T01:52:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:54 smithi159 ceph-mon[67110]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T01:52:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:54 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3111052990' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-19T01:52:54.961 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.962+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 --> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fe108061dc0 con 0x7fe0f00a28a0 2024-03-19T01:52:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.981+0000 7fe1057fa640 1 -- 172.21.15.92:0/783076958 <== mgr.14152 v2:172.21.15.92:6800/3230736370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fe108061dc0 con 0x7fe0f00a28a0 2024-03-19T01:52:54.980 INFO:teuthology.orchestra.run.smithi092.stdout:{} 2024-03-19T01:52:54.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0f00a28a0 msgr2=0x7fe0f00a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0f00a28a0 0x7fe0f00a4d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe108101760 tx=0x7fe0f8005f90 comp rx=0 tx=0).stop 2024-03-19T01:52:54.982 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 msgr2=0x7fe10819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:52:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 0x7fe10819c7f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005940 tx=0x7fe0fc032470 comp rx=0 tx=0).stop 2024-03-19T01:52:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 shutdown_connections 2024-03-19T01:52:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe0f00a28a0 0x7fe0f00a4d60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1081022c0 0x7fe10819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe108101900 0x7fe10819c7f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 --2- 172.21.15.92:0/783076958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe108100710 0x7fe108069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:52:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 >> 172.21.15.92:0/783076958 conn(0x7fe1080fbc80 msgr2=0x7fe108104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:52:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 shutdown_connections 2024-03-19T01:52:54.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T01:52:54.983+0000 7fe10e777640 1 -- 172.21.15.92:0/783076958 wait complete. 2024-03-19T01:52:54.994 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-19T01:52:55.008 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-19T01:52:55.043 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi092.front.sepia.ceph.com 2024-03-19T01:52:55.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-19T01:52:55.045 INFO:teuthology.run_tasks:Running task workunit... 2024-03-19T01:52:55.079 INFO:tasks.workunit:Pulling workunits from ref e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-19T01:52:55.079 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-19T01:52:55.079 INFO:tasks.workunit:timeout=3h 2024-03-19T01:52:55.079 INFO:tasks.workunit:cleanup=True 2024-03-19T01:52:55.080 DEBUG:teuthology.orchestra.run.smithi092:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-19T01:52:55.096 INFO:teuthology.orchestra.run.smithi092.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-19T01:52:55.096 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-19T01:52:55.096 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-19T01:52:55.096 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-19T01:52:55.096 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-19T01:52:55.096 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-19 01:50:57.007265177 +0000 2024-03-19T01:52:55.096 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-19 01:50:57.007265177 +0000 2024-03-19T01:52:55.096 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-19 01:52:53.738063867 +0000 2024-03-19T01:52:55.097 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-03-19 01:50:57.007265177 +0000 2024-03-19T01:52:55.097 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-19T01:52:55.097 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-19T01:52:55.161 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:52:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:52:54.968+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:52:55.161 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:52:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:52:54.968+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:52:55.161 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:52:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:52:54.968+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:52:55.161 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:52:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:52:54.968+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:52:55.161 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 01:52:54 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T01:52:54.968+0000 7fceff957700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-19T01:52:55.172 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e48b1f6c0608b2615e1890cc46286c3877d6e910 2024-03-19T01:52:55.228 INFO:tasks.workunit.client.0.smithi092.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-19T01:52:55.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:55 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3111052990' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-19T01:52:55.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:55 smithi144 ceph-mon[67199]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:52:55.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:55 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3111052990' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-19T01:52:55.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:55 smithi159 ceph-mon[67110]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:52:56.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:55 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3111052990' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-19T01:52:56.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:55 smithi092 ceph-mon[62574]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:52:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:56 smithi159 ceph-mon[67110]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:56 smithi159 ceph-mon[67110]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:56 smithi092 ceph-mon[62574]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:56 smithi092 ceph-mon[62574]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:56 smithi144 ceph-mon[67199]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-19T01:52:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:56 smithi144 ceph-mon[67199]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-19T01:52:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:58 smithi159 ceph-mon[67110]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-19T01:52:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:52:58 smithi159 ceph-mon[67110]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-19T01:52:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:58 smithi092 ceph-mon[62574]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-19T01:52:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:52:58 smithi092 ceph-mon[62574]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-19T01:52:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:58 smithi144 ceph-mon[67199]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-19T01:52:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:52:58 smithi144 ceph-mon[67199]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-19T01:53:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:00 smithi159 ceph-mon[67110]: pgmap v357: 97 pgs: 97 active+clean; 655 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-19T01:53:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:00 smithi092 ceph-mon[62574]: pgmap v357: 97 pgs: 97 active+clean; 655 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-19T01:53:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:00 smithi144 ceph-mon[67199]: pgmap v357: 97 pgs: 97 active+clean; 655 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-19T01:53:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:02 smithi159 ceph-mon[67110]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-19T01:53:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:02 smithi092 ceph-mon[62574]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-19T01:53:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:02 smithi144 ceph-mon[67199]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-19T01:53:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:04 smithi159 ceph-mon[67110]: pgmap v359: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-03-19T01:53:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:04 smithi092 ceph-mon[62574]: pgmap v359: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-03-19T01:53:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:04 smithi144 ceph-mon[67199]: pgmap v359: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-03-19T01:53:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:06 smithi159 ceph-mon[67110]: pgmap v360: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-03-19T01:53:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:06 smithi092 ceph-mon[62574]: pgmap v360: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-03-19T01:53:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:06 smithi144 ceph-mon[67199]: pgmap v360: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-03-19T01:53:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.190+0000 7faaa3387640 1 -- 172.21.15.92:0/2761075861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c101930 msgr2=0x7faa9c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.190+0000 7faaa3387640 1 --2- 172.21.15.92:0/2761075861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c101930 0x7faa9c101db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009900 tx=0x7faa8c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:53:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.190+0000 7faaa3387640 1 -- 172.21.15.92:0/2761075861 shutdown_connections 2024-03-19T01:53:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.190+0000 7faaa3387640 1 --2- 172.21.15.92:0/2761075861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa9c1022f0 0x7faa9c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.190+0000 7faaa3387640 1 --2- 172.21.15.92:0/2761075861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c101930 0x7faa9c101db0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.190+0000 7faaa3387640 1 --2- 172.21.15.92:0/2761075861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa9c100740 0x7faa9c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.190+0000 7faaa3387640 1 -- 172.21.15.92:0/2761075861 >> 172.21.15.92:0/2761075861 conn(0x7faa9c0fbcb0 msgr2=0x7faa9c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.190+0000 7faaa3387640 1 -- 172.21.15.92:0/2761075861 shutdown_connections 2024-03-19T01:53:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.190+0000 7faaa3387640 1 -- 172.21.15.92:0/2761075861 wait complete. 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa3387640 1 Processor -- start 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa3387640 1 -- start start 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa3387640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa9c100740 0x7faa9c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa3387640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa9c101930 0x7faa9c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa3387640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c1022f0 0x7faa9c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa3387640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10c3c0 con 0x7faa9c1022f0 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa3387640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10c240 con 0x7faa9c100740 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa3387640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10c540 con 0x7faa9c101930 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa1924640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c1022f0 0x7faa9c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa0922640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa9c101930 0x7faa9c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa1924640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c1022f0 0x7faa9c19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36648/0 (socket says 172.21.15.92:36648) 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa0922640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa9c101930 0x7faa9c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:36856/0 (socket says 172.21.15.92:36856) 2024-03-19T01:53:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa1924640 1 -- 172.21.15.92:0/524362323 learned_addr learned my addr 172.21.15.92:0/524362323 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.191+0000 7faaa1123640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa9c100740 0x7faa9c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa1924640 1 -- 172.21.15.92:0/524362323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa9c101930 msgr2=0x7faa9c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa1924640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa9c101930 0x7faa9c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa1924640 1 -- 172.21.15.92:0/524362323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa9c100740 msgr2=0x7faa9c069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa1924640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa9c100740 0x7faa9c069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa1924640 1 -- 172.21.15.92:0/524362323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c19f720 con 0x7faa9c1022f0 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa1123640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa9c100740 0x7faa9c069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa0922640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa9c101930 0x7faa9c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa1924640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c1022f0 0x7faa9c19ee10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7faa9400eb10 tx=0x7faa9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faa927fc640 1 -- 172.21.15.92:0/524362323 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94004b20 con 0x7faa9c1022f0 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa3387640 1 -- 172.21.15.92:0/524362323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c196620 con 0x7faa9c1022f0 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faa927fc640 1 -- 172.21.15.92:0/524362323 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa94004cc0 con 0x7faa9c1022f0 2024-03-19T01:53:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faaa3387640 1 -- 172.21.15.92:0/524362323 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c196b90 con 0x7faa9c1022f0 2024-03-19T01:53:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.192+0000 7faa927fc640 1 -- 172.21.15.92:0/524362323 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94010760 con 0x7faa9c1022f0 2024-03-19T01:53:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faa927fc640 1 -- 172.21.15.92:0/524362323 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7faa94005020 con 0x7faa9c1022f0 2024-03-19T01:53:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faaa5f73740 1 Processor -- start 2024-03-19T01:53:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faaa5f73740 1 -- start start 2024-03-19T01:53:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 0x5646a1ba2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba3300 0x5646a1ba7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba8470 0x5646a1baa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faaa5f73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5646a1afebe0 con 0x5646a1ba8470 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faaa5f73740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5646a1aa7960 con 0x5646a1ba3300 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faaa5f73740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5646a1afac80 con 0x5646a1ba2910 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.195+0000 7faaa0922640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba8470 0x5646a1baa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa0922640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba8470 0x5646a1baa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36664/0 (socket says 172.21.15.92:36664) 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa0922640 1 -- 172.21.15.92:0/1205150840 learned_addr learned my addr 172.21.15.92:0/1205150840 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa1123640 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba3300 0x5646a1ba7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa0922640 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 0x5646a1ba2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faa927fc640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7faa880a2940 0x7faa880a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa1924640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7faa880a2940 0x7faa880a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa1123640 1 -- 172.21.15.92:0/1205150840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 msgr2=0x5646a1ba2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa1123640 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 0x5646a1ba2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa1123640 1 -- 172.21.15.92:0/1205150840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba8470 msgr2=0x5646a1baa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa1123640 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba8470 0x5646a1baa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa1123640 1 -- 172.21.15.92:0/1205150840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a1a9ed20 con 0x5646a1ba3300 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa0922640 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba8470 0x5646a1baa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faa927fc640 1 -- 172.21.15.92:0/524362323 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7faa940cf2f0 con 0x7faa9c1022f0 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa1123640 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba3300 0x5646a1ba7e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa98009fa0 tx=0x7faa980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b58bbc74fbfc90f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faaa1924640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7faa880a2940 0x7faa880a4e00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa94098510 tx=0x7faa940058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faa877fe640 1 -- 172.21.15.92:0/1205150840 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98004310 con 0x5646a1ba3300 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faa877fe640 1 -- 172.21.15.92:0/1205150840 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa980044b0 con 0x5646a1ba3300 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.196+0000 7faa877fe640 1 -- 172.21.15.92:0/1205150840 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa980035f0 con 0x5646a1ba3300 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 -- 172.21.15.92:0/1205150840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba3300 msgr2=0x5646a1ba7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba3300 0x5646a1ba7e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa98009fa0 tx=0x7faa980118d0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 -- 172.21.15.92:0/1205150840 shutdown_connections 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba8470 0x5646a1baa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba3300 0x5646a1ba7e00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1205150840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 0x5646a1ba2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 -- 172.21.15.92:0/1205150840 >> 172.21.15.92:0/1205150840 conn(0x5646a1b95670 msgr2=0x5646a1b95a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 -- 172.21.15.92:0/1205150840 shutdown_connections 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 -- 172.21.15.92:0/1205150840 wait complete. 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.197+0000 7faaa5f73740 1 -- start start 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa5f73740 1 Processor -- start 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 0x5646a1bac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 0x5646a1bb3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa5f73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5646a1bab420 con 0x5646a1ba3300 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa5f73740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5646a1abadb0 con 0x5646a1ba8470 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa5f73740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5646a1bab5a0 con 0x5646a1ba2910 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa5f73740 10 client.0 _ll_register_callbacks cb 0x5646a1abfe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa1123640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa1123640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bace80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36678/0 (socket says 172.21.15.92:36678) 2024-03-19T01:53:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa1123640 1 -- 172.21.15.92:0/1994895816 learned_addr learned my addr 172.21.15.92:0/1994895816 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa0922640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 0x5646a1bac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa0922640 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 0x5646a1bb3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa1123640 1 -- 172.21.15.92:0/1994895816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 msgr2=0x5646a1bac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa1123640 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 0x5646a1bac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa1123640 1 -- 172.21.15.92:0/1994895816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 msgr2=0x5646a1bb3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa1123640 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 0x5646a1bb3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.198+0000 7faaa1123640 1 -- 172.21.15.92:0/1994895816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a1ab9c60 con 0x5646a1ba3300 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faaa1123640 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bace80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faa9801b040 tx=0x7faa98009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a034f5a5690ac68 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faa84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faa84ff9640 1 -- 172.21.15.92:0/1994895816 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98028070 con 0x5646a1ba3300 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faa84ff9640 1 -- 172.21.15.92:0/1994895816 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7faa980037f0 con 0x5646a1ba3300 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faaa5f73740 1 -- 172.21.15.92:0/1994895816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa980044b0 con 0x5646a1ba3300 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faa84ff9640 1 -- 172.21.15.92:0/1994895816 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa98033050 con 0x5646a1ba3300 2024-03-19T01:53:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faaa5f73740 10 client.15336 fetch_fsmap learned FSMap version 27 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faaa5f73740 1 -- 172.21.15.92:0/1994895816 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646a1813210 con 0x5646a1ba3300 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faa84ff9640 1 -- 172.21.15.92:0/1994895816 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa980044b0 con 0x5646a1ba3300 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faa84ff9640 1 -- 172.21.15.92:0/1994895816 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faa9800f4e0 con 0x5646a1ba3300 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faaa5f73740 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.199+0000 7faaa5f73740 10 client.15336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.200+0000 7faaa5f73740 20 client.15336 populate_metadata read hostname 'smithi092' 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.200+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5646a1bb8530 0x5646a1bba970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.200+0000 7faaa5f73740 4 client.15336 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.200+0000 7faaa5f73740 1 -- 172.21.15.92:0/1994895816 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646a1b0c120 con 0x5646a1bb8530 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.200+0000 7faaa0922640 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5646a1bb8530 0x5646a1bba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.200+0000 7faa84ff9640 10 client.15336 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.200+0000 7faaa0922640 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5646a1bb8530 0x5646a1bba970 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a5a3bb54aa58cd6 server_cookie=5c7ad5e616f82d97 in_seq=0 out_seq=0 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faa84ff9640 1 -- 172.21.15.92:0/1994895816 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5646a1b0c120 con 0x5646a1bb8530 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faa84ff9640 10 client.15336 handle_command_reply: tid=0 2024-03-19T01:53:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faaa5f73740 1 client.15336 shutdown 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faaa5f73740 1 -- 172.21.15.92:0/1994895816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 msgr2=0x5646a1bace80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bace80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faa9801b040 tx=0x7faa98009d10 comp rx=0 tx=0).stop 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faaa5f73740 1 -- 172.21.15.92:0/1994895816 shutdown_connections 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5646a1bb8530 0x5646a1bba970 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 0x5646a1bb3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faa84ff9640 0 client.15336 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bace80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1994895816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba2910 0x5646a1bac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.201+0000 7faaa5f73740 1 -- 172.21.15.92:0/1994895816 >> 172.21.15.92:0/1994895816 conn(0x5646a1b95670 msgr2=0x5646a1b968c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.202+0000 7faaa5f73740 1 -- 172.21.15.92:0/1994895816 shutdown_connections 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.202+0000 7faaa5f73740 1 -- 172.21.15.92:0/1994895816 wait complete. 2024-03-19T01:53:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.202+0000 7faaa5f73740 20 client.15336 trim_cache size 0 max 16384 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.223+0000 7faaa5f73740 1 Processor -- start 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.223+0000 7faaa5f73740 1 -- start start 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.223+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba2910 0x5646a1b94ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.223+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba3300 0x5646a1bb3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.223+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 0x5646a1ba7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.223+0000 7faaa5f73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c0724a0 con 0x5646a1ba2910 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.223+0000 7faaa5f73740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c19fe80 con 0x5646a1ba8470 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.223+0000 7faaa5f73740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c072b70 con 0x5646a1ba3300 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa0922640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba2910 0x5646a1b94ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa1123640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba3300 0x5646a1bb3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa1123640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba3300 0x5646a1bb3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:36902/0 (socket says 172.21.15.92:36902) 2024-03-19T01:53:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa0922640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba2910 0x5646a1b94ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36690/0 (socket says 172.21.15.92:36690) 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa1123640 1 -- 172.21.15.92:0/1036257887 learned_addr learned my addr 172.21.15.92:0/1036257887 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa0922640 1 --2- 172.21.15.92:0/1036257887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 0x5646a1ba7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa1123640 1 -- 172.21.15.92:0/1036257887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 msgr2=0x5646a1ba7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa1123640 1 --2- 172.21.15.92:0/1036257887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 0x5646a1ba7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa1123640 1 -- 172.21.15.92:0/1036257887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba2910 msgr2=0x5646a1b94ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa1123640 1 --2- 172.21.15.92:0/1036257887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba2910 0x5646a1b94ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa1123640 1 -- 172.21.15.92:0/1036257887 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a18df620 con 0x5646a1ba3300 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa0922640 1 --2- 172.21.15.92:0/1036257887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba2910 0x5646a1b94ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faaa1123640 1 --2- 172.21.15.92:0/1036257887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba3300 0x5646a1bb3e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faa98009230 tx=0x7faa980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd7e76657d26f967 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faa86ffd640 1 -- 172.21.15.92:0/1036257887 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98030070 con 0x5646a1ba3300 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faa86ffd640 1 -- 172.21.15.92:0/1036257887 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa980035f0 con 0x5646a1ba3300 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.224+0000 7faa86ffd640 1 -- 172.21.15.92:0/1036257887 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98028070 con 0x5646a1ba3300 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 -- 172.21.15.92:0/1036257887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba3300 msgr2=0x5646a1bb3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1036257887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba3300 0x5646a1bb3e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faa98009230 tx=0x7faa980118d0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 -- 172.21.15.92:0/1036257887 shutdown_connections 2024-03-19T01:53:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1036257887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba8470 0x5646a1ba7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1036257887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba3300 0x5646a1bb3e80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 --2- 172.21.15.92:0/1036257887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba2910 0x5646a1b94ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 -- 172.21.15.92:0/1036257887 >> 172.21.15.92:0/1036257887 conn(0x5646a1b95670 msgr2=0x5646a1b988d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 -- 172.21.15.92:0/1036257887 shutdown_connections 2024-03-19T01:53:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 -- 172.21.15.92:0/1036257887 wait complete. 2024-03-19T01:53:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 -- start start 2024-03-19T01:53:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.225+0000 7faaa5f73740 1 Processor -- start 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba2910 0x5646a1bb19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bb1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa5f73740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba8470 0x5646a1bb2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa5f73740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c19f8f0 con 0x5646a1ba3300 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa5f73740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c073240 con 0x5646a1ba2910 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa5f73740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c070480 con 0x5646a1ba8470 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa5f73740 10 client.0 _ll_register_callbacks cb 0x7faa940093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa1123640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bb1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa1123640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bb1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36696/0 (socket says 172.21.15.92:36696) 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa1123640 1 -- 172.21.15.92:0/584374960 learned_addr learned my addr 172.21.15.92:0/584374960 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa0922640 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba8470 0x5646a1bb2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa0922640 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba2910 0x5646a1bb19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa1123640 1 -- 172.21.15.92:0/584374960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba8470 msgr2=0x5646a1bb2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa1123640 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba8470 0x5646a1bb2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa1123640 1 -- 172.21.15.92:0/584374960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba2910 msgr2=0x5646a1bb19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa1123640 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba2910 0x5646a1bb19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa1123640 1 -- 172.21.15.92:0/584374960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a1bb3530 con 0x5646a1ba3300 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faaa1123640 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bb1f30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa98006010 tx=0x7faa9800ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f452be8af77b08d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.226+0000 7faa84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faa84ff9640 1 -- 172.21.15.92:0/584374960 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98030030 con 0x5646a1ba3300 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faa84ff9640 1 -- 172.21.15.92:0/584374960 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7faa98004500 con 0x5646a1ba3300 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faaa5f73740 1 -- 172.21.15.92:0/584374960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa980035f0 con 0x5646a1ba3300 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faa84ff9640 1 -- 172.21.15.92:0/584374960 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa98028040 con 0x5646a1ba3300 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faaa5f73740 10 client.15342 fetch_fsmap learned FSMap version 27 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faaa5f73740 1 -- 172.21.15.92:0/584374960 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646a1c1ec00 con 0x5646a1ba3300 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faa84ff9640 1 -- 172.21.15.92:0/584374960 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa980035f0 con 0x5646a1ba3300 2024-03-19T01:53:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faa84ff9640 1 -- 172.21.15.92:0/584374960 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faa980387e0 con 0x5646a1ba3300 2024-03-19T01:53:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faaa5f73740 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T01:53:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faaa5f73740 10 client.15342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:53:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faaa5f73740 20 client.15342 populate_metadata read hostname 'smithi092' 2024-03-19T01:53:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faaa5f73740 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5646a1c1f4c0 0x5646a1c218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faaa5f73740 4 client.15342 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:53:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.227+0000 7faaa5f73740 1 -- 172.21.15.92:0/584374960 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646a1b06470 con 0x5646a1c1f4c0 2024-03-19T01:53:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.228+0000 7faaa0922640 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5646a1c1f4c0 0x5646a1c218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.228+0000 7faaa0922640 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5646a1c1f4c0 0x5646a1c218e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a782abc24a3619 server_cookie=aebdd025a467714 in_seq=0 out_seq=0 2024-03-19T01:53:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.228+0000 7faa84ff9640 10 client.15342 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faa84ff9640 1 -- 172.21.15.92:0/584374960 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646a1b06470 con 0x5646a1c1f4c0 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faa84ff9640 10 client.15342 handle_command_reply: tid=0 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faaa5f73740 1 client.15342 shutdown 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faaa5f73740 1 -- 172.21.15.92:0/584374960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 msgr2=0x5646a1bb1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faaa5f73740 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bb1f30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa98006010 tx=0x7faa9800ffb0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faaa5f73740 1 -- 172.21.15.92:0/584374960 shutdown_connections 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faaa5f73740 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5646a1c1f4c0 0x5646a1c218e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faaa5f73740 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5646a1ba8470 0x5646a1bb2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faaa5f73740 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5646a1ba3300 0x5646a1bb1f30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faaa5f73740 1 --2- 172.21.15.92:0/584374960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a1ba2910 0x5646a1bb19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faaa5f73740 1 -- 172.21.15.92:0/584374960 >> 172.21.15.92:0/584374960 conn(0x5646a1b95670 msgr2=0x5646a1b99160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.238+0000 7faa84ff9640 0 client.15342 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa5f73740 1 -- 172.21.15.92:0/584374960 shutdown_connections 2024-03-19T01:53:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa5f73740 1 -- 172.21.15.92:0/584374960 wait complete. 2024-03-19T01:53:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa5f73740 20 client.15342 trim_cache size 0 max 16384 2024-03-19T01:53:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:53:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:53:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:53:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:53:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 -- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7faa880a2940 msgr2=0x7faa880a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7faa880a2940 0x7faa880a4e00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa94098510 tx=0x7faa940058d0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 -- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c1022f0 msgr2=0x7faa9c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c1022f0 0x7faa9c19ee10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7faa9400eb10 tx=0x7faa9400c2d0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 -- 172.21.15.92:0/524362323 shutdown_connections 2024-03-19T01:53:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa9c1022f0 0x7faa9c19ee10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7faa880a2940 0x7faa880a4e00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa9c101930 0x7faa9c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 --2- 172.21.15.92:0/524362323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa9c100740 0x7faa9c069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 -- 172.21.15.92:0/524362323 >> 172.21.15.92:0/524362323 conn(0x7faa9c0fbcb0 msgr2=0x7faa9c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 -- 172.21.15.92:0/524362323 shutdown_connections 2024-03-19T01:53:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:08.239+0000 7faaa3387640 1 -- 172.21.15.92:0/524362323 wait complete. 2024-03-19T01:53:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:08 smithi159 ceph-mon[67110]: pgmap v361: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-19T01:53:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:08 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:53:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:08 smithi092 ceph-mon[62574]: pgmap v361: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-19T01:53:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:08 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:53:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:08 smithi144 ceph-mon[67199]: pgmap v361: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-19T01:53:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:08 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:53:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:10 smithi159 ceph-mon[67110]: pgmap v362: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-03-19T01:53:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:10.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:10.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:10.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:10 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:10 smithi092 ceph-mon[62574]: pgmap v362: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-03-19T01:53:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:10 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:10 smithi144 ceph-mon[67199]: pgmap v362: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-03-19T01:53:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:10 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:11.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:53:11.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:53:11.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:11 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:12.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:12.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:12.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:53:12.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:53:12.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:11 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:12.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:12.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:12.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:53:12.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:53:12.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:11 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:53:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:12 smithi159 ceph-mon[67110]: pgmap v363: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-19T01:53:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:12 smithi092 ceph-mon[62574]: pgmap v363: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-19T01:53:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:12 smithi144 ceph-mon[67199]: pgmap v363: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-19T01:53:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:14 smithi159 ceph-mon[67110]: pgmap v364: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-19T01:53:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:14 smithi092 ceph-mon[62574]: pgmap v364: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-19T01:53:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:14 smithi144 ceph-mon[67199]: pgmap v364: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 16 op/s 2024-03-19T01:53:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:16 smithi159 ceph-mon[67110]: pgmap v365: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-03-19T01:53:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:16 smithi092 ceph-mon[62574]: pgmap v365: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-03-19T01:53:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:16 smithi144 ceph-mon[67199]: pgmap v365: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-03-19T01:53:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:18 smithi159 ceph-mon[67110]: pgmap v366: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-03-19T01:53:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:18 smithi092 ceph-mon[62574]: pgmap v366: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-03-19T01:53:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:18 smithi144 ceph-mon[67199]: pgmap v366: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-03-19T01:53:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:20 smithi159 ceph-mon[67110]: pgmap v367: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-03-19T01:53:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:20 smithi092 ceph-mon[62574]: pgmap v367: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-03-19T01:53:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:20 smithi144 ceph-mon[67199]: pgmap v367: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-03-19T01:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.362+0000 7f515a1d4640 1 -- 172.21.15.92:0/1317182692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154107560 msgr2=0x7f5154109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.362+0000 7f515a1d4640 1 --2- 172.21.15.92:0/1317182692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154107560 0x7f5154109970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f514c009be0 tx=0x7f514c02bef0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.362+0000 7f515a1d4640 1 -- 172.21.15.92:0/1317182692 shutdown_connections 2024-03-19T01:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.362+0000 7f515a1d4640 1 --2- 172.21.15.92:0/1317182692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154107560 0x7f5154109970 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.362+0000 7f515a1d4640 1 --2- 172.21.15.92:0/1317182692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5154104c30 0x7f5154107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.362+0000 7f515a1d4640 1 --2- 172.21.15.92:0/1317182692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5154102300 0x7f51541046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.362+0000 7f515a1d4640 1 -- 172.21.15.92:0/1317182692 >> 172.21.15.92:0/1317182692 conn(0x7f51540fbcb0 msgr2=0x7f51540fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.362+0000 7f515a1d4640 1 -- 172.21.15.92:0/1317182692 shutdown_connections 2024-03-19T01:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.362+0000 7f515a1d4640 1 -- 172.21.15.92:0/1317182692 wait complete. 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.363+0000 7f515a1d4640 1 Processor -- start 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.363+0000 7f515a1d4640 1 -- start start 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.363+0000 7f515a1d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5154102300 0x7f5154197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.363+0000 7f515a1d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154104c30 0x7f5154198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f515a1d4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5154107560 0x7f515419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f515a1d4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d510 con 0x7f5154102300 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f515a1d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d390 con 0x7f5154104c30 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f515a1d4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d690 con 0x7f5154107560 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f51537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5154102300 0x7f5154197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f51537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5154102300 0x7f5154197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33258/0 (socket says 172.21.15.92:33258) 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f51537fe640 1 -- 172.21.15.92:0/3275391122 learned_addr learned my addr 172.21.15.92:0/3275391122 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f5152ffd640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154104c30 0x7f5154198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f5153fff640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5154107560 0x7f515419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f5152ffd640 1 -- 172.21.15.92:0/3275391122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5154107560 msgr2=0x7f515419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f5152ffd640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5154107560 0x7f515419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f5152ffd640 1 -- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5154102300 msgr2=0x7f5154197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f5152ffd640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5154102300 0x7f5154197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f5152ffd640 1 -- 172.21.15.92:0/3275391122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515419f6d0 con 0x7f5154104c30 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f51537fe640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5154102300 0x7f5154197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f5152ffd640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154104c30 0x7f5154198040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5148005fd0 tx=0x7f514800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.364+0000 7f5150ff9640 1 -- 172.21.15.92:0/3275391122 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800ccc0 con 0x7f5154104c30 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.365+0000 7f5150ff9640 1 -- 172.21.15.92:0/3275391122 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514800ee70 con 0x7f5154104c30 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.365+0000 7f5150ff9640 1 -- 172.21.15.92:0/3275391122 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51480187e0 con 0x7f5154104c30 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.366+0000 7f515a1d4640 1 -- 172.21.15.92:0/3275391122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515419f9c0 con 0x7f5154104c30 2024-03-19T01:53:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.366+0000 7f515a1d4640 1 -- 172.21.15.92:0/3275391122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515419fec0 con 0x7f5154104c30 2024-03-19T01:53:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.368+0000 7f5150ff9640 1 -- 172.21.15.92:0/3275391122 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f514801f020 con 0x7f5154104c30 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.369+0000 7f515cdc0740 1 Processor -- start 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.369+0000 7f515cdc0740 1 -- start start 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.369+0000 7f5150ff9640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f513c0a29c0 0x7f513c0a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.369+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba61520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.369+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 0x56311ba665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f5150ff9640 1 -- 172.21.15.92:0/3275391122 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f51480ceb70 con 0x7f5154104c30 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f515cdc0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56311b9c5be0 con 0x56311ba61af0 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f515cdc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56311b96e960 con 0x56311ba66c60 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f515cdc0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56311b9c1c80 con 0x56311ba61100 2024-03-19T01:53:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f51537fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba61520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f51537fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba61520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:55166/0 (socket says 172.21.15.92:55166) 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 0x56311ba665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f51537fe640 1 -- 172.21.15.92:0/3819646315 learned_addr learned my addr 172.21.15.92:0/3819646315 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f5153fff640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f513c0a29c0 0x7f513c0a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f51537fe640 1 -- 172.21.15.92:0/3819646315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 msgr2=0x56311ba690f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f51537fe640 1 --2- 172.21.15.92:0/3819646315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba690f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f51537fe640 1 -- 172.21.15.92:0/3819646315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 msgr2=0x56311ba665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f51537fe640 1 --2- 172.21.15.92:0/3819646315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 0x56311ba665f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.370+0000 7f51537fe640 1 -- 172.21.15.92:0/3819646315 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311b965d20 con 0x56311ba61100 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.371+0000 7f5152ffd640 1 --2- 172.21.15.92:0/3819646315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 0x56311ba665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.371+0000 7f51537fe640 1 --2- 172.21.15.92:0/3819646315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba61520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5144011650 tx=0x7f5144009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2123b3d4d9585e37 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.371+0000 7f5139ffb640 1 -- 172.21.15.92:0/3819646315 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514401e070 con 0x56311ba61100 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.371+0000 7f5139ffb640 1 -- 172.21.15.92:0/3819646315 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514400fba0 con 0x56311ba61100 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.371+0000 7f5139ffb640 1 -- 172.21.15.92:0/3819646315 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144019cd0 con 0x56311ba61100 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.371+0000 7f5153fff640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f513c0a29c0 0x7f513c0a4e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f514c006010 tx=0x7f514c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 -- 172.21.15.92:0/3819646315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 msgr2=0x56311ba61520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 --2- 172.21.15.92:0/3819646315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba61520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5144011650 tx=0x7f5144009f90 comp rx=0 tx=0).stop 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 -- 172.21.15.92:0/3819646315 shutdown_connections 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 --2- 172.21.15.92:0/3819646315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 --2- 172.21.15.92:0/3819646315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 0x56311ba665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 --2- 172.21.15.92:0/3819646315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba61520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 -- 172.21.15.92:0/3819646315 >> 172.21.15.92:0/3819646315 conn(0x56311ba5c670 msgr2=0x56311ba5ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 -- 172.21.15.92:0/3819646315 shutdown_connections 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 -- 172.21.15.92:0/3819646315 wait complete. 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 -- start start 2024-03-19T01:53:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.372+0000 7f515cdc0740 1 Processor -- start 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba6b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba61af0 0x56311ba6b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba66c60 0x56311ba72630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f515cdc0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56311ba69c10 con 0x56311ba61100 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f515cdc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56311b981db0 con 0x56311ba61af0 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f515cdc0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56311ba69d90 con 0x56311ba66c60 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f515cdc0740 10 client.0 _ll_register_callbacks cb 0x56311b986e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba61af0 0x56311ba6b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f51537fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba6b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba61af0 0x56311ba6b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:35894/0 (socket says 172.21.15.92:35894) 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5152ffd640 1 -- 172.21.15.92:0/2683990375 learned_addr learned my addr 172.21.15.92:0/2683990375 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5153fff640 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba66c60 0x56311ba72630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5152ffd640 1 -- 172.21.15.92:0/2683990375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba66c60 msgr2=0x56311ba72630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5152ffd640 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba66c60 0x56311ba72630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5152ffd640 1 -- 172.21.15.92:0/2683990375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 msgr2=0x56311ba6b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5152ffd640 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba6b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5152ffd640 1 -- 172.21.15.92:0/2683990375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311b980c60 con 0x56311ba61af0 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f5153fff640 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba66c60 0x56311ba72630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.373+0000 7f51537fe640 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba6b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.374+0000 7f5152ffd640 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba61af0 0x56311ba6b5f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f514809b0c0 tx=0x7f514800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c27a6b64dc6a76 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.374+0000 7f5132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.374+0000 7f5132ffd640 1 -- 172.21.15.92:0/2683990375 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514809f510 con 0x56311ba61af0 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.374+0000 7f5132ffd640 1 -- 172.21.15.92:0/2683990375 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f514805cc30 con 0x56311ba61af0 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.374+0000 7f515cdc0740 1 -- 172.21.15.92:0/2683990375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f514400fba0 con 0x56311ba61af0 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.374+0000 7f5132ffd640 1 -- 172.21.15.92:0/2683990375 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51480a3b60 con 0x56311ba61af0 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.374+0000 7f515cdc0740 10 client.25184 fetch_fsmap learned FSMap version 27 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.374+0000 7f515cdc0740 1 -- 172.21.15.92:0/2683990375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56311b6da210 con 0x56311ba61af0 2024-03-19T01:53:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.374+0000 7f5132ffd640 1 -- 172.21.15.92:0/2683990375 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514400fba0 con 0x56311ba61af0 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f5132ffd640 1 -- 172.21.15.92:0/2683990375 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f51480bb0b0 con 0x56311ba61af0 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f515cdc0740 10 client.25184 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f515cdc0740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f515cdc0740 20 client.25184 populate_metadata read hostname 'smithi092' 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56311ba741c0 0x56311ba76680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f515cdc0740 4 client.25184 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f515cdc0740 1 -- 172.21.15.92:0/2683990375 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56311b9d3120 con 0x56311ba741c0 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f51537fe640 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56311ba741c0 0x56311ba76680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f51537fe640 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56311ba741c0 0x56311ba76680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1039b89e3a2889f7 server_cookie=dfd1ed3b9f17bd6b in_seq=0 out_seq=0 2024-03-19T01:53:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.375+0000 7f5132ffd640 10 client.25184 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f5132ffd640 1 -- 172.21.15.92:0/2683990375 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56311b9d3120 con 0x56311ba741c0 2024-03-19T01:53:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f5132ffd640 10 client.25184 handle_command_reply: tid=0 2024-03-19T01:53:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 client.25184 shutdown 2024-03-19T01:53:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 -- 172.21.15.92:0/2683990375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba61af0 msgr2=0x56311ba6b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba61af0 0x56311ba6b5f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f514809b0c0 tx=0x7f514800cf20 comp rx=0 tx=0).stop 2024-03-19T01:53:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 -- 172.21.15.92:0/2683990375 shutdown_connections 2024-03-19T01:53:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56311ba741c0 0x56311ba76680 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f5132ffd640 0 client.25184 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba66c60 0x56311ba72630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba61af0 0x56311ba6b5f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2683990375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba6b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 -- 172.21.15.92:0/2683990375 >> 172.21.15.92:0/2683990375 conn(0x56311ba5c670 msgr2=0x56311ba5d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 -- 172.21.15.92:0/2683990375 shutdown_connections 2024-03-19T01:53:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 1 -- 172.21.15.92:0/2683990375 wait complete. 2024-03-19T01:53:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.376+0000 7f515cdc0740 20 client.25184 trim_cache size 0 max 16384 2024-03-19T01:53:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.391+0000 7f515cdc0740 1 Processor -- start 2024-03-19T01:53:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.391+0000 7f515cdc0740 1 -- start start 2024-03-19T01:53:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.391+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba5b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61af0 0x56311ba725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f515cdc0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f51541a6ad0 con 0x56311ba61100 2024-03-19T01:53:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f515cdc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f51541a8610 con 0x56311ba66c60 2024-03-19T01:53:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f515cdc0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51541a71a0 con 0x56311ba61af0 2024-03-19T01:53:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61af0 0x56311ba725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61af0 0x56311ba725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:55174/0 (socket says 172.21.15.92:55174) 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5152ffd640 1 -- 172.21.15.92:0/3299847258 learned_addr learned my addr 172.21.15.92:0/3299847258 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f51537fe640 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba5b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5153fff640 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5152ffd640 1 -- 172.21.15.92:0/3299847258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 msgr2=0x56311ba5b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5152ffd640 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba5b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5152ffd640 1 -- 172.21.15.92:0/3299847258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 msgr2=0x56311ba5b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5152ffd640 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba5b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5152ffd640 1 -- 172.21.15.92:0/3299847258 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311b7a6620 con 0x56311ba61af0 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f51537fe640 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba5b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.392+0000 7f5152ffd640 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61af0 0x56311ba725f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5148000c00 tx=0x7f514800cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5befafc43a790b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.393+0000 7f51397fa640 1 -- 172.21.15.92:0/3299847258 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51480b4240 con 0x56311ba61af0 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.393+0000 7f51397fa640 1 -- 172.21.15.92:0/3299847258 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51480bb040 con 0x56311ba61af0 2024-03-19T01:53:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.393+0000 7f51397fa640 1 -- 172.21.15.92:0/3299847258 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148018980 con 0x56311ba61af0 2024-03-19T01:53:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.393+0000 7f515cdc0740 1 -- 172.21.15.92:0/3299847258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61af0 msgr2=0x56311ba725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.393+0000 7f515cdc0740 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61af0 0x56311ba725f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5148000c00 tx=0x7f514800cf90 comp rx=0 tx=0).stop 2024-03-19T01:53:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.394+0000 7f515cdc0740 1 -- 172.21.15.92:0/3299847258 shutdown_connections 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.394+0000 7f515cdc0740 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.394+0000 7f515cdc0740 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61af0 0x56311ba725f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.394+0000 7f515cdc0740 1 --2- 172.21.15.92:0/3299847258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61100 0x56311ba5b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.394+0000 7f515cdc0740 1 -- 172.21.15.92:0/3299847258 >> 172.21.15.92:0/3299847258 conn(0x56311ba5c670 msgr2=0x56311ba5ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.394+0000 7f515cdc0740 1 -- 172.21.15.92:0/3299847258 shutdown_connections 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.394+0000 7f515cdc0740 1 -- 172.21.15.92:0/3299847258 wait complete. 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.394+0000 7f515cdc0740 1 -- start start 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.394+0000 7f515cdc0740 1 Processor -- start 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba6fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 0x56311ba70260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f515cdc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba71200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f515cdc0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f51541a7f40 con 0x56311ba61af0 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f51537fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba6fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f515cdc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f51541a7870 con 0x56311ba66c60 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f515cdc0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51541a4ab0 con 0x56311ba61100 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5153fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba71200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5153fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba71200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:35912/0 (socket says 172.21.15.92:35912) 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 0x56311ba70260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f51537fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba6fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:55190/0 (socket says 172.21.15.92:55190) 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5153fff640 1 -- 172.21.15.92:0/2810349518 learned_addr learned my addr 172.21.15.92:0/2810349518 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f515cdc0740 10 client.0 _ll_register_callbacks cb 0x7f51480520b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5153fff640 1 -- 172.21.15.92:0/2810349518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 msgr2=0x56311ba6fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5153fff640 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba6fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5153fff640 1 -- 172.21.15.92:0/2810349518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 msgr2=0x56311ba70260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5153fff640 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 0x56311ba70260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5153fff640 1 -- 172.21.15.92:0/2810349518 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311ba717f0 con 0x56311ba66c60 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.395+0000 7f5153fff640 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba71200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f514c009be0 tx=0x7f514c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5607540c9e4ef07 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.396+0000 7f5132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.396+0000 7f5132ffd640 1 -- 172.21.15.92:0/2810349518 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c02cd20 con 0x56311ba66c60 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.396+0000 7f5132ffd640 1 -- 172.21.15.92:0/2810349518 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f514c003fe0 con 0x56311ba66c60 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.396+0000 7f515cdc0740 1 -- 172.21.15.92:0/2810349518 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51480bb040 con 0x56311ba66c60 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.396+0000 7f5132ffd640 1 -- 172.21.15.92:0/2810349518 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f514c0041e0 con 0x56311ba66c60 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.396+0000 7f515cdc0740 10 client.25190 fetch_fsmap learned FSMap version 27 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.396+0000 7f515cdc0740 1 -- 172.21.15.92:0/2810349518 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56311bae5ac0 con 0x56311ba66c60 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.396+0000 7f5132ffd640 1 -- 172.21.15.92:0/2810349518 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514c034430 con 0x56311ba66c60 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f5132ffd640 1 -- 172.21.15.92:0/2810349518 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f514c03f440 con 0x56311ba66c60 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f515cdc0740 10 client.25190 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f515cdc0740 10 client.25190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f515cdc0740 20 client.25190 populate_metadata read hostname 'smithi092' 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56311ba741c0 0x56311bae8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f515cdc0740 4 client.25190 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:53:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f515cdc0740 1 -- 172.21.15.92:0/2810349518 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f51480bb040 con 0x56311ba741c0 2024-03-19T01:53:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f51537fe640 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56311ba741c0 0x56311bae8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f5132ffd640 10 client.25190 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.397+0000 7f51537fe640 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56311ba741c0 0x56311bae8340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=108f10557eb6cbda server_cookie=26ef6668c29bbef1 in_seq=0 out_seq=0 2024-03-19T01:53:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.400+0000 7f5132ffd640 1 -- 172.21.15.92:0/2810349518 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f51480bb040 con 0x56311ba741c0 2024-03-19T01:53:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.400+0000 7f5132ffd640 10 client.25190 handle_command_reply: tid=0 2024-03-19T01:53:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.400+0000 7f515cdc0740 1 client.25190 shutdown 2024-03-19T01:53:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 -- 172.21.15.92:0/2810349518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 msgr2=0x56311ba71200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba71200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f514c009be0 tx=0x7f514c009500 comp rx=0 tx=0).stop 2024-03-19T01:53:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 -- 172.21.15.92:0/2810349518 shutdown_connections 2024-03-19T01:53:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56311ba741c0 0x56311bae8340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56311ba66c60 0x56311ba71200 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f5132ffd640 0 client.25190 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56311ba61af0 0x56311ba70260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 --2- 172.21.15.92:0/2810349518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56311ba61100 0x56311ba6fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 -- 172.21.15.92:0/2810349518 >> 172.21.15.92:0/2810349518 conn(0x56311ba5c670 msgr2=0x56311ba6aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 -- 172.21.15.92:0/2810349518 shutdown_connections 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 1 -- 172.21.15.92:0/2810349518 wait complete. 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.401+0000 7f515cdc0740 20 client.25190 trim_cache size 0 max 16384 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:53:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:53:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 -- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f513c0a29c0 msgr2=0x7f513c0a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f513c0a29c0 0x7f513c0a4e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f514c006010 tx=0x7f514c004a40 comp rx=0 tx=0).stop 2024-03-19T01:53:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 -- 172.21.15.92:0/3275391122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154104c30 msgr2=0x7f5154198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154104c30 0x7f5154198040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5148005fd0 tx=0x7f514800ec20 comp rx=0 tx=0).stop 2024-03-19T01:53:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 -- 172.21.15.92:0/3275391122 shutdown_connections 2024-03-19T01:53:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f513c0a29c0 0x7f513c0a4e80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5154107560 0x7f515419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5154104c30 0x7f5154198040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 --2- 172.21.15.92:0/3275391122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5154102300 0x7f5154197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.402+0000 7f515a1d4640 1 -- 172.21.15.92:0/3275391122 >> 172.21.15.92:0/3275391122 conn(0x7f51540fbcb0 msgr2=0x7f5154102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.403+0000 7f515a1d4640 1 -- 172.21.15.92:0/3275391122 shutdown_connections 2024-03-19T01:53:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:21.403+0000 7f515a1d4640 1 -- 172.21.15.92:0/3275391122 wait complete. 2024-03-19T01:53:22.705 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:22 smithi159 ceph-mon[67110]: pgmap v368: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-19T01:53:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:22 smithi092 ceph-mon[62574]: pgmap v368: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-19T01:53:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:22 smithi144 ceph-mon[67199]: pgmap v368: 97 pgs: 97 active+clean; 658 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-19T01:53:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:24 smithi159 ceph-mon[67110]: pgmap v369: 97 pgs: 97 active+clean; 659 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 13 op/s 2024-03-19T01:53:25.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:24 smithi092 ceph-mon[62574]: pgmap v369: 97 pgs: 97 active+clean; 659 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 13 op/s 2024-03-19T01:53:25.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:24 smithi144 ceph-mon[67199]: pgmap v369: 97 pgs: 97 active+clean; 659 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 13 op/s 2024-03-19T01:53:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:26 smithi159 ceph-mon[67110]: pgmap v370: 97 pgs: 97 active+clean; 659 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:53:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:26 smithi092 ceph-mon[62574]: pgmap v370: 97 pgs: 97 active+clean; 659 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:53:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:26 smithi144 ceph-mon[67199]: pgmap v370: 97 pgs: 97 active+clean; 659 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:53:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:28 smithi159 ceph-mon[67110]: pgmap v371: 97 pgs: 97 active+clean; 659 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:53:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:28 smithi092 ceph-mon[62574]: pgmap v371: 97 pgs: 97 active+clean; 659 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:53:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:28 smithi144 ceph-mon[67199]: pgmap v371: 97 pgs: 97 active+clean; 659 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T01:53:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:30 smithi159 ceph-mon[67110]: pgmap v372: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-03-19T01:53:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:30 smithi092 ceph-mon[62574]: pgmap v372: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-03-19T01:53:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:30 smithi144 ceph-mon[67199]: pgmap v372: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-03-19T01:53:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:32 smithi159 ceph-mon[67110]: pgmap v373: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-19T01:53:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:32 smithi092 ceph-mon[62574]: pgmap v373: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-19T01:53:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:32 smithi144 ceph-mon[67199]: pgmap v373: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-03-19T01:53:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.504+0000 7f1009edc640 1 -- 172.21.15.92:0/3473062616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 msgr2=0x7f1004100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.504+0000 7f1009edc640 1 --2- 172.21.15.92:0/3473062616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 0x7f1004100b60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009900 tx=0x7f0ff402bc30 comp rx=0 tx=0).stop 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.504+0000 7f1009edc640 1 -- 172.21.15.92:0/3473062616 shutdown_connections 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.504+0000 7f1009edc640 1 --2- 172.21.15.92:0/3473062616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10041022f0 0x7f1004108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.504+0000 7f1009edc640 1 --2- 172.21.15.92:0/3473062616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1004101930 0x7f1004101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.504+0000 7f1009edc640 1 --2- 172.21.15.92:0/3473062616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 0x7f1004100b60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.504+0000 7f1009edc640 1 -- 172.21.15.92:0/3473062616 >> 172.21.15.92:0/3473062616 conn(0x7f10040fbcb0 msgr2=0x7f10040fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.504+0000 7f1009edc640 1 -- 172.21.15.92:0/3473062616 shutdown_connections 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.504+0000 7f1009edc640 1 -- 172.21.15.92:0/3473062616 wait complete. 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.505+0000 7f1009edc640 1 Processor -- start 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.505+0000 7f1009edc640 1 -- start start 2024-03-19T01:53:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f1009edc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 0x7f1004069a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f1009edc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1004101930 0x7f100419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f1009edc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10041022f0 0x7f100419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f10037fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 0x7f1004069a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f10037fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 0x7f1004069a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46672/0 (socket says 172.21.15.92:46672) 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f10037fe640 1 -- 172.21.15.92:0/1453777094 learned_addr learned my addr 172.21.15.92:0/1453777094 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f100410c400 con 0x7f1004100740 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f1003fff640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10041022f0 0x7f100419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f1002ffd640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1004101930 0x7f100419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f100410c280 con 0x7f1004101930 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f100410c580 con 0x7f10041022f0 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f10037fe640 1 -- 172.21.15.92:0/1453777094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10041022f0 msgr2=0x7f100419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f10037fe640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10041022f0 0x7f100419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f10037fe640 1 -- 172.21.15.92:0/1453777094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1004101930 msgr2=0x7f100419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f10037fe640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1004101930 0x7f100419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f10037fe640 1 -- 172.21.15.92:0/1453777094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100419f6f0 con 0x7f1004100740 2024-03-19T01:53:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.506+0000 7f1002ffd640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1004101930 0x7f100419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.507+0000 7f10037fe640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 0x7f1004069a30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4000c00 tx=0x7f0ff402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.507+0000 7f1000ff9640 1 -- 172.21.15.92:0/1453777094 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4043020 con 0x7f1004100740 2024-03-19T01:53:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.507+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100419f980 con 0x7f1004100740 2024-03-19T01:53:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.507+0000 7f1000ff9640 1 -- 172.21.15.92:0/1453777094 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff40054e0 con 0x7f1004100740 2024-03-19T01:53:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.507+0000 7f1000ff9640 1 -- 172.21.15.92:0/1453777094 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4043020 con 0x7f1004100740 2024-03-19T01:53:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.507+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100419fe50 con 0x7f1004100740 2024-03-19T01:53:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.510+0000 7f1000ff9640 1 -- 172.21.15.92:0/1453777094 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f0ff4046020 con 0x7f1004100740 2024-03-19T01:53:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.510+0000 7f1000ff9640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f0fec0a2b30 0x7f0fec0a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.510+0000 7f1000ff9640 1 -- 172.21.15.92:0/1453777094 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0ff40f07b0 con 0x7f1004100740 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.510+0000 7f1002ffd640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f0fec0a2b30 0x7f0fec0a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f100cba7740 1 Processor -- start 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f100cba7740 1 -- start start 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f100cba7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e3f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f100cba7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e3faf0 0x5574f0e445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f100cba7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e44c60 0x5574f0e470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f100cba7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5574f0da3be0 con 0x5574f0e3faf0 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f100cba7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574f0d4c960 con 0x5574f0e3f100 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f100cba7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5574f0d9fc80 con 0x5574f0e44c60 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f1002ffd640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f0fec0a2b30 0x7f0fec0a4ff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f10040697a0 tx=0x7f0ff8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f1003fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e44c60 0x5574f0e470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f10037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e3f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f1003fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e44c60 0x5574f0e470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:35636/0 (socket says 172.21.15.92:35636) 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f1003fff640 1 -- 172.21.15.92:0/2038546129 learned_addr learned my addr 172.21.15.92:0/2038546129 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f1002ffd640 1 --2- 172.21.15.92:0/2038546129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e3faf0 0x5574f0e445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f10037fe640 1 -- 172.21.15.92:0/2038546129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e44c60 msgr2=0x5574f0e470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f10037fe640 1 --2- 172.21.15.92:0/2038546129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e44c60 0x5574f0e470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f10037fe640 1 -- 172.21.15.92:0/2038546129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e3faf0 msgr2=0x5574f0e445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f10037fe640 1 --2- 172.21.15.92:0/2038546129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e3faf0 0x5574f0e445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f10037fe640 1 -- 172.21.15.92:0/2038546129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f0d43d20 con 0x5574f0e3f100 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.511+0000 7f1002ffd640 1 --2- 172.21.15.92:0/2038546129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e3faf0 0x5574f0e445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.512+0000 7f10037fe640 1 --2- 172.21.15.92:0/2038546129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e3f520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40ee280 tx=0x7f0ff40b93b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d094594834a700b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.512+0000 7f0fe9ffb640 1 -- 172.21.15.92:0/2038546129 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40bd110 con 0x5574f0e3f100 2024-03-19T01:53:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.512+0000 7f0fe9ffb640 1 -- 172.21.15.92:0/2038546129 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff40c1260 con 0x5574f0e3f100 2024-03-19T01:53:34.515 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.512+0000 7f0fe9ffb640 1 -- 172.21.15.92:0/2038546129 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40bd370 con 0x5574f0e3f100 2024-03-19T01:53:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 -- 172.21.15.92:0/2038546129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 msgr2=0x5574f0e3f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 --2- 172.21.15.92:0/2038546129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e3f520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40ee280 tx=0x7f0ff40b93b0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 -- 172.21.15.92:0/2038546129 shutdown_connections 2024-03-19T01:53:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 --2- 172.21.15.92:0/2038546129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e44c60 0x5574f0e470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 --2- 172.21.15.92:0/2038546129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e3faf0 0x5574f0e445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 --2- 172.21.15.92:0/2038546129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e3f520 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 -- 172.21.15.92:0/2038546129 >> 172.21.15.92:0/2038546129 conn(0x5574f0e3a670 msgr2=0x5574f0e3aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 -- 172.21.15.92:0/2038546129 shutdown_connections 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 -- 172.21.15.92:0/2038546129 wait complete. 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.513+0000 7f100cba7740 1 -- start start 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f100cba7740 1 Processor -- start 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f100cba7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f100cba7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f100cba7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e54710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f10037fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f10037fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:35652/0 (socket says 172.21.15.92:35652) 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f10037fe640 1 -- 172.21.15.92:0/597865765 learned_addr learned my addr 172.21.15.92:0/597865765 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5574f0e4bde0 con 0x5574f0e44c60 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f1003fff640 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e54710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574f0d5fdb0 con 0x5574f0e3faf0 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5574f0e4bf60 con 0x5574f0e3f100 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f100cba7740 10 client.0 _ll_register_callbacks cb 0x5574f0d64e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f1002ffd640 1 -- 172.21.15.92:0/597865765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3f100 msgr2=0x5574f0e4d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f1002ffd640 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f1002ffd640 1 -- 172.21.15.92:0/597865765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 msgr2=0x5574f0e54710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f1002ffd640 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e54710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f1002ffd640 1 -- 172.21.15.92:0/597865765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f0d5ec60 con 0x5574f0e3faf0 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f10037fe640 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f1003fff640 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e54710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f1002ffd640 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5574f0e40560 tx=0x7f0ff8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=932eeb688e08e839 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.514+0000 7f0fe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/597865765 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800fad0 con 0x5574f0e3faf0 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/597865765 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0ff800fc70 con 0x5574f0e3faf0 2024-03-19T01:53:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff40c1260 con 0x5574f0e3faf0 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/597865765 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ff800fe70 con 0x5574f0e3faf0 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f100cba7740 10 client.25202 fetch_fsmap learned FSMap version 27 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5574f0ab8210 con 0x5574f0e3faf0 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/597865765 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff802c070 con 0x5574f0e3faf0 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/597865765 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ff802f070 con 0x5574f0e3faf0 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f100cba7740 10 client.25202 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f100cba7740 10 client.25202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f100cba7740 20 client.25202 populate_metadata read hostname 'smithi092' 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f100cba7740 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5574f0e58fe0 0x5574f0e5b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f100cba7740 4 client.25202 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.515+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0ff40c1260 con 0x5574f0e58fe0 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.516+0000 7f10037fe640 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5574f0e58fe0 0x5574f0e5b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.516+0000 7f0fe2ffd640 10 client.25202 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.516+0000 7f10037fe640 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5574f0e58fe0 0x5574f0e5b460 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591e92e468038a9 server_cookie=bc929d0f88c32399 in_seq=0 out_seq=0 2024-03-19T01:53:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/597865765 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f0ff40c1260 con 0x5574f0e58fe0 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f0fe2ffd640 10 client.25202 handle_command_reply: tid=0 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 client.25202 shutdown 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3faf0 msgr2=0x5574f0e4d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5574f0e40560 tx=0x7f0ff8004730 comp rx=0 tx=0).stop 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 shutdown_connections 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5574f0e58fe0 0x5574f0e5b460 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e54710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f0fe2ffd640 0 client.25202 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 --2- 172.21.15.92:0/597865765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 >> 172.21.15.92:0/597865765 conn(0x5574f0e3a670 msgr2=0x5574f0e3b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 shutdown_connections 2024-03-19T01:53:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 1 -- 172.21.15.92:0/597865765 wait complete. 2024-03-19T01:53:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.517+0000 7f100cba7740 20 client.25202 trim_cache size 0 max 16384 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.545+0000 7f100cba7740 1 Processor -- start 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.545+0000 7f100cba7740 1 -- start start 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.545+0000 7f100cba7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.545+0000 7f100cba7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.545+0000 7f100cba7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f10037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f10037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:57216/0 (socket says 172.21.15.92:57216) 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f1003fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:35666/0 (socket says 172.21.15.92:35666) 2024-03-19T01:53:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f1003fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46698/0 (socket says 172.21.15.92:46698) 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f100cba7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10041a6a80 con 0x5574f0e44c60 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f10037fe640 1 -- 172.21.15.92:0/888479512 learned_addr learned my addr 172.21.15.92:0/888479512 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f100cba7740 1 -- 172.21.15.92:0/888479512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10041a85c0 con 0x5574f0e3f100 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f100cba7740 1 -- 172.21.15.92:0/888479512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f10041a7150 con 0x5574f0e3faf0 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f10037fe640 1 -- 172.21.15.92:0/888479512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 msgr2=0x5574f0e4d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f10037fe640 1 --2- 172.21.15.92:0/888479512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f10037fe640 1 -- 172.21.15.92:0/888479512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 msgr2=0x5574f0e546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f10037fe640 1 --2- 172.21.15.92:0/888479512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f1002ffd640 1 --2- 172.21.15.92:0/888479512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f10037fe640 1 -- 172.21.15.92:0/888479512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f0b84620 con 0x5574f0e3f100 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f1003fff640 1 --2- 172.21.15.92:0/888479512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.546+0000 7f10037fe640 1 --2- 172.21.15.92:0/888479512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40b9a90 tx=0x7f0ff40b94e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8defcacebc44b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.547+0000 7f0fe97fa640 1 -- 172.21.15.92:0/888479512 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40c1260 con 0x5574f0e3f100 2024-03-19T01:53:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.547+0000 7f0fe97fa640 1 -- 172.21.15.92:0/888479512 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff40ee3c0 con 0x5574f0e3f100 2024-03-19T01:53:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.547+0000 7f0fe97fa640 1 -- 172.21.15.92:0/888479512 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40bd5d0 con 0x5574f0e3f100 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 -- 172.21.15.92:0/888479512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 msgr2=0x5574f0e4d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 --2- 172.21.15.92:0/888479512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40b9a90 tx=0x7f0ff40b94e0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 -- 172.21.15.92:0/888479512 shutdown_connections 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 --2- 172.21.15.92:0/888479512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 --2- 172.21.15.92:0/888479512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e4d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 --2- 172.21.15.92:0/888479512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e4d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 -- 172.21.15.92:0/888479512 >> 172.21.15.92:0/888479512 conn(0x5574f0e3a670 msgr2=0x5574f0e3aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 -- 172.21.15.92:0/888479512 shutdown_connections 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 -- 172.21.15.92:0/888479512 wait complete. 2024-03-19T01:53:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.548+0000 7f100cba7740 1 -- start start 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f100cba7740 1 Processor -- start 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f100cba7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e51c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f100cba7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e52170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f100cba7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e53180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f10037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e51c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f10037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e51c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:57218/0 (socket says 172.21.15.92:57218) 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f10037fe640 1 -- 172.21.15.92:0/2947190828 learned_addr learned my addr 172.21.15.92:0/2947190828 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f1002ffd640 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e52170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10041a7ef0 con 0x5574f0e44c60 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10041a7820 con 0x5574f0e3f100 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f1003fff640 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e53180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f10041a4a60 con 0x5574f0e3faf0 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.549+0000 7f100cba7740 10 client.0 _ll_register_callbacks cb 0x7f0ff8004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f10037fe640 1 -- 172.21.15.92:0/2947190828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 msgr2=0x5574f0e52170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f10037fe640 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e52170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f10037fe640 1 -- 172.21.15.92:0/2947190828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 msgr2=0x5574f0e53180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f10037fe640 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e53180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f10037fe640 1 -- 172.21.15.92:0/2947190828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f0e53770 con 0x5574f0e3f100 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f1002ffd640 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e52170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f10037fe640 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e51c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40ee300 tx=0x7f0ff40ee1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be83f6a0995c7e27 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f0fe2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/2947190828 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40c1260 con 0x5574f0e3f100 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/2947190828 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0ff40bd580 con 0x5574f0e3f100 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5574f0db1120 con 0x5574f0e3f100 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/2947190828 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ff40c5420 con 0x5574f0e3f100 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f100cba7740 10 client.25208 fetch_fsmap learned FSMap version 27 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5574f0e54a60 con 0x5574f0e3f100 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.550+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/2947190828 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5574f0db1120 con 0x5574f0e3f100 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/2947190828 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ff40a6100 con 0x5574f0e3f100 2024-03-19T01:53:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f100cba7740 10 client.25208 fetch_fsmap finished waiting for FSMap version 27 2024-03-19T01:53:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f100cba7740 10 client.25208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:53:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f100cba7740 20 client.25208 populate_metadata read hostname 'smithi092' 2024-03-19T01:53:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f100cba7740 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5574f0e58fe0 0x5574f0ec16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f100cba7740 4 client.25208 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:53:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574f0dab470 con 0x5574f0e58fe0 2024-03-19T01:53:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f1002ffd640 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5574f0e58fe0 0x5574f0ec16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f0fe2ffd640 10 client.25208 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.551+0000 7f1002ffd640 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5574f0e58fe0 0x5574f0ec16a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ff98d6071b0b72 server_cookie=813d4d943a6d78e4 in_seq=0 out_seq=0 2024-03-19T01:53:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.558+0000 7f0fe2ffd640 1 -- 172.21.15.92:0/2947190828 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574f0dab470 con 0x5574f0e58fe0 2024-03-19T01:53:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.558+0000 7f0fe2ffd640 10 client.25208 handle_command_reply: tid=0 2024-03-19T01:53:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.558+0000 7f100cba7740 1 client.25208 shutdown 2024-03-19T01:53:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 msgr2=0x5574f0e51c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e51c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40ee300 tx=0x7f0ff40ee1c0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 shutdown_connections 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5574f0e58fe0 0x5574f0ec16a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574f0e44c60 0x5574f0e53180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574f0e3faf0 0x5574f0e52170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 --2- 172.21.15.92:0/2947190828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574f0e3f100 0x5574f0e51c30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 >> 172.21.15.92:0/2947190828 conn(0x5574f0e3a670 msgr2=0x5574f0e4d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f0fe2ffd640 0 client.25208 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 shutdown_connections 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 1 -- 172.21.15.92:0/2947190828 wait complete. 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.559+0000 7f100cba7740 20 client.25208 trim_cache size 0 max 16384 2024-03-19T01:53:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:53:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:53:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:53:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:53:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f0fec0a2b30 msgr2=0x7f0fec0a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f0fec0a2b30 0x7f0fec0a4ff0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f10040697a0 tx=0x7f0ff8005f90 comp rx=0 tx=0).stop 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 msgr2=0x7f1004069a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 0x7f1004069a30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4000c00 tx=0x7f0ff402c6c0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 shutdown_connections 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f0fec0a2b30 0x7f0fec0a4ff0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10041022f0 0x7f100419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1004101930 0x7f100419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 --2- 172.21.15.92:0/1453777094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1004100740 0x7f1004069a30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 >> 172.21.15.92:0/1453777094 conn(0x7f10040fbcb0 msgr2=0x7f1004104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 shutdown_connections 2024-03-19T01:53:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:34.560+0000 7f1009edc640 1 -- 172.21.15.92:0/1453777094 wait complete. 2024-03-19T01:53:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:34 smithi159 ceph-mon[67110]: pgmap v374: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-19T01:53:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:34 smithi092 ceph-mon[62574]: pgmap v374: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-19T01:53:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:34 smithi144 ceph-mon[67199]: pgmap v374: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-19T01:53:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:36 smithi159 ceph-mon[67110]: pgmap v375: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-19T01:53:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:36 smithi092 ceph-mon[62574]: pgmap v375: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-19T01:53:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:36 smithi144 ceph-mon[67199]: pgmap v375: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-19T01:53:38.184 INFO:tasks.workunit.client.0.smithi092.stderr:Note: switching to 'e48b1f6c0608b2615e1890cc46286c3877d6e910'. 2024-03-19T01:53:38.184 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-03-19T01:53:38.184 INFO:tasks.workunit.client.0.smithi092.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-19T01:53:38.184 INFO:tasks.workunit.client.0.smithi092.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-19T01:53:38.184 INFO:tasks.workunit.client.0.smithi092.stderr:state without impacting any branches by switching back to a branch. 2024-03-19T01:53:38.184 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-03-19T01:53:38.184 INFO:tasks.workunit.client.0.smithi092.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-19T01:53:38.184 INFO:tasks.workunit.client.0.smithi092.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-19T01:53:38.184 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-03-19T01:53:38.185 INFO:tasks.workunit.client.0.smithi092.stderr: git switch -c 2024-03-19T01:53:38.185 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-03-19T01:53:38.185 INFO:tasks.workunit.client.0.smithi092.stderr:Or undo this operation with: 2024-03-19T01:53:38.185 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-03-19T01:53:38.185 INFO:tasks.workunit.client.0.smithi092.stderr: git switch - 2024-03-19T01:53:38.185 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-03-19T01:53:38.185 INFO:tasks.workunit.client.0.smithi092.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-19T01:53:38.185 INFO:tasks.workunit.client.0.smithi092.stderr: 2024-03-19T01:53:38.185 INFO:tasks.workunit.client.0.smithi092.stderr:HEAD is now at e48b1f6c060 testfix 2024-03-19T01:53:38.189 DEBUG:teuthology.orchestra.run.smithi092:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-19T01:53:38.252 INFO:tasks.workunit.client.0.smithi092.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-19T01:53:38.254 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-19T01:53:38.254 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-19T01:53:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:38 smithi159 ceph-mon[67110]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-19T01:53:39.020 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-19T01:53:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:38 smithi092 ceph-mon[62574]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-19T01:53:39.080 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-19T01:53:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:38 smithi144 ceph-mon[67199]: pgmap v376: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-19T01:53:39.119 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-19T01:53:39.121 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-19T01:53:39.121 INFO:tasks.workunit.client.0.smithi092.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-19T01:53:39.160 INFO:tasks.workunit.client.0.smithi092.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-19T01:53:39.164 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T01:53:39.164 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-19T01:53:39.219 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-03-19T01:53:39.221 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-03-19T01:53:39.221 DEBUG:teuthology.orchestra.run.smithi092:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e48b1f6c0608b2615e1890cc46286c3877d6e910 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-03-19T01:53:39.282 INFO:tasks.workunit.client.0.smithi092.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-03-19T01:53:39.282 INFO:tasks.workunit.client.0.smithi092.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-03-19T01:53:39.282 INFO:tasks.workunit.client.0.smithi092.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-03-19T01:53:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:40 smithi159 ceph-mon[67110]: pgmap v377: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-19T01:53:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:40 smithi092 ceph-mon[62574]: pgmap v377: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-19T01:53:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:40 smithi144 ceph-mon[67199]: pgmap v377: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-19T01:53:41.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:41 smithi159 ceph-mon[67110]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:53:42.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:41 smithi092 ceph-mon[62574]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:53:42.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:41 smithi144 ceph-mon[67199]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:53:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:42 smithi159 ceph-mon[67110]: pgmap v378: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-03-19T01:53:43.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:42 smithi092 ceph-mon[62574]: pgmap v378: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-03-19T01:53:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:42 smithi144 ceph-mon[67199]: pgmap v378: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-03-19T01:53:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:44 smithi159 ceph-mon[67110]: pgmap v379: 97 pgs: 97 active+clean; 708 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-03-19T01:53:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:44 smithi092 ceph-mon[62574]: pgmap v379: 97 pgs: 97 active+clean; 708 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-03-19T01:53:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:44 smithi144 ceph-mon[67199]: pgmap v379: 97 pgs: 97 active+clean; 708 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-03-19T01:53:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:46 smithi159 ceph-mon[67110]: pgmap v380: 97 pgs: 97 active+clean; 708 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-03-19T01:53:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:46 smithi092 ceph-mon[62574]: pgmap v380: 97 pgs: 97 active+clean; 708 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-03-19T01:53:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:46 smithi144 ceph-mon[67199]: pgmap v380: 97 pgs: 97 active+clean; 708 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-03-19T01:53:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.647+0000 7fe1a3480640 1 -- 172.21.15.92:0/2402064565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c101d40 msgr2=0x7fe19c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.647+0000 7fe1a3480640 1 --2- 172.21.15.92:0/2402064565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c101d40 0x7fe19c102160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe194009900 tx=0x7fe19402bc30 comp rx=0 tx=0).stop 2024-03-19T01:53:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.647+0000 7fe1a3480640 1 -- 172.21.15.92:0/2402064565 shutdown_connections 2024-03-19T01:53:47.650 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.647+0000 7fe1a3480640 1 --2- 172.21.15.92:0/2402064565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c100280 0x7fe19c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.647+0000 7fe1a3480640 1 --2- 172.21.15.92:0/2402064565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19c0ff8c0 0x7fe19c0ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.647+0000 7fe1a3480640 1 --2- 172.21.15.92:0/2402064565 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c101d40 0x7fe19c102160 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.647+0000 7fe1a3480640 1 -- 172.21.15.92:0/2402064565 >> 172.21.15.92:0/2402064565 conn(0x7fe19c0fb440 msgr2=0x7fe19c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.648+0000 7fe1a3480640 1 -- 172.21.15.92:0/2402064565 shutdown_connections 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.648+0000 7fe1a3480640 1 -- 172.21.15.92:0/2402064565 wait complete. 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.648+0000 7fe1a3480640 1 Processor -- start 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.648+0000 7fe1a3480640 1 -- start start 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a3480640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19c0ff8c0 0x7fe19c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a3480640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c100280 0x7fe19c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a3480640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c101d40 0x7fe19c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a3480640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10f6c0 con 0x7fe19c100280 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a3480640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10f540 con 0x7fe19c0ff8c0 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a3480640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10f840 con 0x7fe19c101d40 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a0a1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c100280 0x7fe19c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a0a1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c100280 0x7fe19c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46598/0 (socket says 172.21.15.92:46598) 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a0a1b640 1 -- 172.21.15.92:0/554993077 learned_addr learned my addr 172.21.15.92:0/554993077 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a121c640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19c0ff8c0 0x7fe19c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c101d40 0x7fe19c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a0a1b640 1 -- 172.21.15.92:0/554993077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c101d40 msgr2=0x7fe19c1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a0a1b640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c101d40 0x7fe19c1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a0a1b640 1 -- 172.21.15.92:0/554993077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19c0ff8c0 msgr2=0x7fe19c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a0a1b640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19c0ff8c0 0x7fe19c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a0a1b640 1 -- 172.21.15.92:0/554993077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c1a1900 con 0x7fe19c100280 2024-03-19T01:53:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a0a1b640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c100280 0x7fe19c19a230 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe19000ebc0 tx=0x7fe19000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe19a7fc640 1 -- 172.21.15.92:0/554993077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190010630 con 0x7fe19c100280 2024-03-19T01:53:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe19a7fc640 1 -- 172.21.15.92:0/554993077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe190010cb0 con 0x7fe19c100280 2024-03-19T01:53:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a3480640 1 -- 172.21.15.92:0/554993077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c1a1bf0 con 0x7fe19c100280 2024-03-19T01:53:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe19a7fc640 1 -- 172.21.15.92:0/554993077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190018bd0 con 0x7fe19c100280 2024-03-19T01:53:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.649+0000 7fe1a3480640 1 -- 172.21.15.92:0/554993077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c1a2140 con 0x7fe19c100280 2024-03-19T01:53:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a6060740 1 Processor -- start 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe19a7fc640 1 -- 172.21.15.92:0/554993077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fe1900107d0 con 0x7fe19c100280 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a6060740 1 -- start start 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cabf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cac45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 0x55937cac70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a6060740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55937ca23be0 con 0x55937cabf100 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a6060740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55937c9cc960 con 0x55937cac4c60 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a6060740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55937ca1fc80 con 0x55937cabfaf0 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a1a1d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cabf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a1a1d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cabf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46602/0 (socket says 172.21.15.92:46602) 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/3757543225 learned_addr learned my addr 172.21.15.92:0/3757543225 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a121c640 1 --2- 172.21.15.92:0/3757543225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cac45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.652+0000 7fe1a0a1b640 1 --2- 172.21.15.92:0/3757543225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 0x55937cac70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/3757543225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 msgr2=0x55937cac45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/3757543225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cac45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/3757543225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 msgr2=0x55937cac70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/3757543225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 0x55937cac70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/3757543225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937c9c3d20 con 0x55937cabf100 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe19a7fc640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1840a2940 0x7fe1840a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1a121c640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1840a2940 0x7fe1840a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe19a7fc640 1 -- 172.21.15.92:0/554993077 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe1900cf250 con 0x7fe19c100280 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/3757543225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cabf520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009ca0 tx=0x7fe18c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c893d585a4568e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1837fe640 1 -- 172.21.15.92:0/3757543225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c01e070 con 0x55937cabf100 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1837fe640 1 -- 172.21.15.92:0/3757543225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c009e50 con 0x55937cabf100 2024-03-19T01:53:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.653+0000 7fe1837fe640 1 -- 172.21.15.92:0/3757543225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00ea00 con 0x55937cabf100 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 -- 172.21.15.92:0/3757543225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 msgr2=0x55937cabf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a121c640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1840a2940 0x7fe1840a4e00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe19c19b1c0 tx=0x7fe194005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3757543225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cabf520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009ca0 tx=0x7fe18c011810 comp rx=0 tx=0).stop 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 -- 172.21.15.92:0/3757543225 shutdown_connections 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3757543225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 0x55937cac70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3757543225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cac45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3757543225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cabf520 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 -- 172.21.15.92:0/3757543225 >> 172.21.15.92:0/3757543225 conn(0x55937caba670 msgr2=0x55937cabaa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 -- 172.21.15.92:0/3757543225 shutdown_connections 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 -- 172.21.15.92:0/3757543225 wait complete. 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 -- start start 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.654+0000 7fe1a6060740 1 Processor -- start 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cac9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cac9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 0x55937cad0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a6060740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55937cac7c10 con 0x55937cabf100 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a6060740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55937c9dfdb0 con 0x55937cac4c60 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a6060740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55937cac7d90 con 0x55937cabfaf0 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a121c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cac9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a6060740 10 client.0 _ll_register_callbacks cb 0x55937c9e4e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a121c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cac9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46618/0 (socket says 172.21.15.92:46618) 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a121c640 1 -- 172.21.15.92:0/3600639863 learned_addr learned my addr 172.21.15.92:0/3600639863 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 0x55937cad0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a0a1b640 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cac9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a121c640 1 -- 172.21.15.92:0/3600639863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 msgr2=0x55937cac9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a121c640 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cac9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a121c640 1 -- 172.21.15.92:0/3600639863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 msgr2=0x55937cad0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a121c640 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 0x55937cad0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a121c640 1 -- 172.21.15.92:0/3600639863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937c9dec60 con 0x55937cabf100 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe1a121c640 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cac9110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe194009900 tx=0x7fe194009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed5808e6a81df09 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.655+0000 7fe180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe180ff9640 1 -- 172.21.15.92:0/3600639863 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19404c070 con 0x55937cabf100 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe180ff9640 1 -- 172.21.15.92:0/3600639863 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe194005050 con 0x55937cabf100 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a6060740 1 -- 172.21.15.92:0/3600639863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18c009e50 con 0x55937cabf100 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe180ff9640 1 -- 172.21.15.92:0/3600639863 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe194003e50 con 0x55937cabf100 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a6060740 10 client.15387 fetch_fsmap learned FSMap version 28 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a6060740 1 -- 172.21.15.92:0/3600639863 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55937c738210 con 0x55937cabf100 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe180ff9640 1 -- 172.21.15.92:0/3600639863 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c009e50 con 0x55937cabf100 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe180ff9640 1 -- 172.21.15.92:0/3600639863 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe194051460 con 0x55937cabf100 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a6060740 10 client.15387 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a6060740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a6060740 20 client.15387 populate_metadata read hostname 'smithi092' 2024-03-19T01:53:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55937cad4f70 0x55937cad73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a6060740 4 client.15387 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a6060740 1 -- 172.21.15.92:0/3600639863 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55937ca31120 con 0x55937cad4f70 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.656+0000 7fe1a0a1b640 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55937cad4f70 0x55937cad73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.657+0000 7fe180ff9640 10 client.15387 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.657+0000 7fe1a0a1b640 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55937cad4f70 0x55937cad73e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69bafccdbf6bf099 server_cookie=e3cd7d0f00688f24 in_seq=0 out_seq=0 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.657+0000 7fe180ff9640 1 -- 172.21.15.92:0/3600639863 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55937ca31120 con 0x55937cad4f70 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.657+0000 7fe180ff9640 10 client.15387 handle_command_reply: tid=0 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.657+0000 7fe1a6060740 1 client.15387 shutdown 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 -- 172.21.15.92:0/3600639863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 msgr2=0x55937cac9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cac9110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe194009900 tx=0x7fe194009260 comp rx=0 tx=0).stop 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 -- 172.21.15.92:0/3600639863 shutdown_connections 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55937cad4f70 0x55937cad73e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cac4c60 0x55937cad0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cac9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe180ff9640 0 client.15387 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 --2- 172.21.15.92:0/3600639863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cabf100 0x55937cac9110 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 -- 172.21.15.92:0/3600639863 >> 172.21.15.92:0/3600639863 conn(0x55937caba670 msgr2=0x55937cabb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 -- 172.21.15.92:0/3600639863 shutdown_connections 2024-03-19T01:53:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 1 -- 172.21.15.92:0/3600639863 wait complete. 2024-03-19T01:53:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.658+0000 7fe1a6060740 20 client.15387 trim_cache size 0 max 16384 2024-03-19T01:53:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.672+0000 7fe1a6060740 1 Processor -- start 2024-03-19T01:53:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.672+0000 7fe1a6060740 1 -- start start 2024-03-19T01:53:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.672+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabf100 0x55937cab9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.672+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabfaf0 0x55937cad0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.672+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cabdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.672+0000 7fe1a6060740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1aadb0 con 0x55937cac4c60 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.672+0000 7fe1a6060740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1ac920 con 0x55937cabfaf0 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.672+0000 7fe1a6060740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1ab480 con 0x55937cabf100 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.672+0000 7fe1a1a1d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cabdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a1a1d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cabdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46624/0 (socket says 172.21.15.92:46624) 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/1504763589 learned_addr learned my addr 172.21.15.92:0/1504763589 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a121c640 1 --2- 172.21.15.92:0/1504763589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabf100 0x55937cab9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a0a1b640 1 --2- 172.21.15.92:0/1504763589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabfaf0 0x55937cad0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/1504763589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabf100 msgr2=0x55937cab9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/1504763589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabf100 0x55937cab9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/1504763589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabfaf0 msgr2=0x55937cad0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/1504763589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabfaf0 0x55937cad0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/1504763589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937c804620 con 0x55937cac4c60 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/1504763589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cabdfe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe18c01b000 tx=0x7fe18c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a00b655e19441b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe182ffd640 1 -- 172.21.15.92:0/1504763589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c02a070 con 0x55937cac4c60 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe182ffd640 1 -- 172.21.15.92:0/1504763589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c01f040 con 0x55937cac4c60 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.673+0000 7fe182ffd640 1 -- 172.21.15.92:0/1504763589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c01e030 con 0x55937cac4c60 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 -- 172.21.15.92:0/1504763589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 msgr2=0x55937cabdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 --2- 172.21.15.92:0/1504763589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cabdfe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe18c01b000 tx=0x7fe18c005850 comp rx=0 tx=0).stop 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 -- 172.21.15.92:0/1504763589 shutdown_connections 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 --2- 172.21.15.92:0/1504763589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cabdfe0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 --2- 172.21.15.92:0/1504763589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabfaf0 0x55937cad0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 --2- 172.21.15.92:0/1504763589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabf100 0x55937cab9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 -- 172.21.15.92:0/1504763589 >> 172.21.15.92:0/1504763589 conn(0x55937caba670 msgr2=0x55937cabd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 -- 172.21.15.92:0/1504763589 shutdown_connections 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 -- 172.21.15.92:0/1504763589 wait complete. 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 -- start start 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 Processor -- start 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabf100 0x55937cace200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cace740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cacf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1ac220 con 0x55937cac4c60 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1abb50 con 0x55937cabf100 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.674+0000 7fe1a6060740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1a8d90 con 0x55937cabfaf0 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a6060740 10 client.0 _ll_register_callbacks cb 0x7fe194009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a1a1d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cacf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a1a1d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cacf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46630/0 (socket says 172.21.15.92:46630) 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a121c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabf100 0x55937cace200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/342780749 learned_addr learned my addr 172.21.15.92:0/342780749 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a0a1b640 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cace740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/342780749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 msgr2=0x55937cace740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cace740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/342780749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabf100 msgr2=0x55937cace200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabf100 0x55937cace200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a1a1d640 1 -- 172.21.15.92:0/342780749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937cacfd40 con 0x55937cac4c60 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a1a1d640 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cacf750 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0057c0 tx=0x7fe18c003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e587249b00cfa361 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe180ff9640 1 -- 172.21.15.92:0/342780749 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c032070 con 0x55937cac4c60 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe180ff9640 1 -- 172.21.15.92:0/342780749 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe18c0035f0 con 0x55937cac4c60 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a6060740 1 -- 172.21.15.92:0/342780749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18c01f040 con 0x55937cac4c60 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe180ff9640 1 -- 172.21.15.92:0/342780749 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe18c02a040 con 0x55937cac4c60 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a6060740 10 client.15393 fetch_fsmap learned FSMap version 28 2024-03-19T01:53:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe1a6060740 1 -- 172.21.15.92:0/342780749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55937cb3b790 con 0x55937cac4c60 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.675+0000 7fe180ff9640 1 -- 172.21.15.92:0/342780749 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c01f040 con 0x55937cac4c60 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe180ff9640 1 -- 172.21.15.92:0/342780749 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe18c035020 con 0x55937cac4c60 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe1a6060740 10 client.15393 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe1a6060740 10 client.15393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe1a6060740 20 client.15393 populate_metadata read hostname 'smithi092' 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe1a6060740 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55937cb3bed0 0x55937cb3e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe1a6060740 4 client.15393 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe1a6060740 1 -- 172.21.15.92:0/342780749 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55937ca2b470 con 0x55937cb3bed0 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe1a121c640 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55937cb3bed0 0x55937cb3e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe180ff9640 10 client.15393 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.676+0000 7fe1a121c640 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55937cb3bed0 0x55937cb3e2f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa46af74843b2808 server_cookie=67e0ee6521044afe in_seq=0 out_seq=0 2024-03-19T01:53:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe180ff9640 1 -- 172.21.15.92:0/342780749 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55937ca2b470 con 0x55937cb3bed0 2024-03-19T01:53:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe180ff9640 10 client.15393 handle_command_reply: tid=0 2024-03-19T01:53:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 client.15393 shutdown 2024-03-19T01:53:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 -- 172.21.15.92:0/342780749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 msgr2=0x55937cacf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cacf750 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0057c0 tx=0x7fe18c003a10 comp rx=0 tx=0).stop 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 -- 172.21.15.92:0/342780749 shutdown_connections 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55937cb3bed0 0x55937cb3e2f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55937cac4c60 0x55937cacf750 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe180ff9640 0 client.15393 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937cabfaf0 0x55937cace740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 --2- 172.21.15.92:0/342780749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55937cabf100 0x55937cace200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 -- 172.21.15.92:0/342780749 >> 172.21.15.92:0/342780749 conn(0x55937caba670 msgr2=0x55937cabe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 -- 172.21.15.92:0/342780749 shutdown_connections 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 1 -- 172.21.15.92:0/342780749 wait complete. 2024-03-19T01:53:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.691+0000 7fe1a6060740 20 client.15393 trim_cache size 0 max 16384 2024-03-19T01:53:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:53:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:53:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:53:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 -- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1840a2940 msgr2=0x7fe1840a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1840a2940 0x7fe1840a4e00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe19c19b1c0 tx=0x7fe194005fd0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 -- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c100280 msgr2=0x7fe19c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c100280 0x7fe19c19a230 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe19000ebc0 tx=0x7fe19000c6a0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 -- 172.21.15.92:0/554993077 shutdown_connections 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe1840a2940 0x7fe1840a4e00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c101d40 0x7fe19c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe19c100280 0x7fe19c19a230 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 --2- 172.21.15.92:0/554993077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19c0ff8c0 0x7fe19c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:53:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 -- 172.21.15.92:0/554993077 >> 172.21.15.92:0/554993077 conn(0x7fe19c0fb440 msgr2=0x7fe19c0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:53:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 -- 172.21.15.92:0/554993077 shutdown_connections 2024-03-19T01:53:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:53:47.692+0000 7fe1a3480640 1 -- 172.21.15.92:0/554993077 wait complete. 2024-03-19T01:53:47.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:47 smithi159 ceph-mon[67110]: pgmap v381: 97 pgs: 97 active+clean; 708 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-03-19T01:53:48.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:47 smithi092 ceph-mon[62574]: pgmap v381: 97 pgs: 97 active+clean; 708 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-03-19T01:53:48.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:47 smithi144 ceph-mon[67199]: pgmap v381: 97 pgs: 97 active+clean; 708 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-03-19T01:53:50.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:50 smithi092 ceph-mon[62574]: pgmap v382: 97 pgs: 97 active+clean; 738 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 17 op/s 2024-03-19T01:53:50.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:50 smithi144 ceph-mon[67199]: pgmap v382: 97 pgs: 97 active+clean; 738 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 17 op/s 2024-03-19T01:53:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:50 smithi159 ceph-mon[67110]: pgmap v382: 97 pgs: 97 active+clean; 738 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 17 op/s 2024-03-19T01:53:52.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:52 smithi092 ceph-mon[62574]: pgmap v383: 97 pgs: 97 active+clean; 738 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 15 op/s 2024-03-19T01:53:52.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:52 smithi144 ceph-mon[67199]: pgmap v383: 97 pgs: 97 active+clean; 738 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 15 op/s 2024-03-19T01:53:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:52 smithi159 ceph-mon[67110]: pgmap v383: 97 pgs: 97 active+clean; 738 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 15 op/s 2024-03-19T01:53:54.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:54 smithi092 ceph-mon[62574]: pgmap v384: 97 pgs: 97 active+clean; 763 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-03-19T01:53:54.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:54 smithi144 ceph-mon[67199]: pgmap v384: 97 pgs: 97 active+clean; 763 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-03-19T01:53:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:54 smithi159 ceph-mon[67110]: pgmap v384: 97 pgs: 97 active+clean; 763 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-03-19T01:53:56.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:56 smithi092 ceph-mon[62574]: pgmap v385: 97 pgs: 97 active+clean; 763 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 KiB/s wr, 16 op/s 2024-03-19T01:53:56.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:56 smithi144 ceph-mon[67199]: pgmap v385: 97 pgs: 97 active+clean; 763 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 KiB/s wr, 16 op/s 2024-03-19T01:53:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:56 smithi159 ceph-mon[67110]: pgmap v385: 97 pgs: 97 active+clean; 763 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 KiB/s wr, 16 op/s 2024-03-19T01:53:58.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:53:58 smithi092 ceph-mon[62574]: pgmap v386: 97 pgs: 97 active+clean; 763 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 KiB/s wr, 17 op/s 2024-03-19T01:53:58.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:53:58 smithi144 ceph-mon[67199]: pgmap v386: 97 pgs: 97 active+clean; 763 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 KiB/s wr, 17 op/s 2024-03-19T01:53:58.957 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:53:58 smithi159 ceph-mon[67110]: pgmap v386: 97 pgs: 97 active+clean; 763 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 KiB/s wr, 17 op/s 2024-03-19T01:54:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.778+0000 7fc11a681640 1 -- 172.21.15.92:0/954416051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114102a40 msgr2=0x7fc114102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.778+0000 7fc11a681640 1 --2- 172.21.15.92:0/954416051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114102a40 0x7fc114102ec0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc108009900 tx=0x7fc10802bc30 comp rx=0 tx=0).stop 2024-03-19T01:54:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.778+0000 7fc11a681640 1 -- 172.21.15.92:0/954416051 shutdown_connections 2024-03-19T01:54:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.778+0000 7fc11a681640 1 --2- 172.21.15.92:0/954416051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc114103400 0x7fc114109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.778+0000 7fc11a681640 1 --2- 172.21.15.92:0/954416051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114102a40 0x7fc114102ec0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.778+0000 7fc11a681640 1 --2- 172.21.15.92:0/954416051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114101850 0x7fc114101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.778+0000 7fc11a681640 1 -- 172.21.15.92:0/954416051 >> 172.21.15.92:0/954416051 conn(0x7fc1140fcde0 msgr2=0x7fc1140ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.778+0000 7fc11a681640 1 -- 172.21.15.92:0/954416051 shutdown_connections 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.778+0000 7fc11a681640 1 -- 172.21.15.92:0/954416051 wait complete. 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc11a681640 1 Processor -- start 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc11a681640 1 -- start start 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc11a681640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114101850 0x7fc114197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc11a681640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc114102a40 0x7fc114197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc11a681640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114103400 0x7fc11419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc11a681640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410d4d0 con 0x7fc114101850 2024-03-19T01:54:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc11a681640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410d350 con 0x7fc114103400 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc11a681640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410d650 con 0x7fc114102a40 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114101850 0x7fc114197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114101850 0x7fc114197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39620/0 (socket says 172.21.15.92:39620) 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc113fff640 1 -- 172.21.15.92:0/2459924812 learned_addr learned my addr 172.21.15.92:0/2459924812 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc1137fe640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc114102a40 0x7fc114197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.779+0000 7fc118c1e640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114103400 0x7fc11419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc113fff640 1 -- 172.21.15.92:0/2459924812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc114102a40 msgr2=0x7fc114197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc113fff640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc114102a40 0x7fc114197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc113fff640 1 -- 172.21.15.92:0/2459924812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114103400 msgr2=0x7fc11419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc113fff640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114103400 0x7fc11419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc113fff640 1 -- 172.21.15.92:0/2459924812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11419f6a0 con 0x7fc114101850 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc113fff640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114101850 0x7fc114197a80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc10400e9c0 tx=0x7fc10400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc1117fa640 1 -- 172.21.15.92:0/2459924812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10400cd80 con 0x7fc114101850 2024-03-19T01:54:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc11a681640 1 -- 172.21.15.92:0/2459924812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11419f990 con 0x7fc114101850 2024-03-19T01:54:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc1117fa640 1 -- 172.21.15.92:0/2459924812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1040035f0 con 0x7fc114101850 2024-03-19T01:54:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc1117fa640 1 -- 172.21.15.92:0/2459924812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104010690 con 0x7fc114101850 2024-03-19T01:54:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.780+0000 7fc11a681640 1 -- 172.21.15.92:0/2459924812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11419ff00 con 0x7fc114101850 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc1117fa640 1 -- 172.21.15.92:0/2459924812 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fc1040037a0 con 0x7fc114101850 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc11d26e740 1 Processor -- start 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc11d26e740 1 -- start start 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 0x56511b732930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b737a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b738070 0x56511b73a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc11d26e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56511b692be0 con 0x56511b732f00 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc11d26e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56511b63b960 con 0x56511b732510 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc11d26e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56511b68ec80 con 0x56511b738070 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b737a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b737a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39630/0 (socket says 172.21.15.92:39630) 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc113fff640 1 -- 172.21.15.92:0/4194081000 learned_addr learned my addr 172.21.15.92:0/4194081000 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc1137fe640 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b738070 0x56511b73a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc118c1e640 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 0x56511b732930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.783+0000 7fc1117fa640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc0fc0a2940 0x7fc0fc0a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc113fff640 1 -- 172.21.15.92:0/4194081000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b738070 msgr2=0x56511b73a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc113fff640 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b738070 0x56511b73a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc113fff640 1 -- 172.21.15.92:0/4194081000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 msgr2=0x56511b732930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc113fff640 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 0x56511b732930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc113fff640 1 -- 172.21.15.92:0/4194081000 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56511b632d20 con 0x56511b732f00 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc1137fe640 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b738070 0x56511b73a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc118c1e640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc0fc0a2940 0x7fc0fc0a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc1117fa640 1 -- 172.21.15.92:0/2459924812 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc1040cef90 con 0x7fc114101850 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc113fff640 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b737a00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc1040cc890 tx=0x7fc1040665a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b650c0ee062f027 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc0fa7fc640 1 -- 172.21.15.92:0/4194081000 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1040af070 con 0x56511b732f00 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc118c1e640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc0fc0a2940 0x7fc0fc0a4e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009fd0 tx=0x7fc10c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc0fa7fc640 1 -- 172.21.15.92:0/4194081000 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10406fd00 con 0x56511b732f00 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.784+0000 7fc0fa7fc640 1 -- 172.21.15.92:0/4194081000 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10406a450 con 0x56511b732f00 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 -- 172.21.15.92:0/4194081000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 msgr2=0x56511b737a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b737a00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc1040cc890 tx=0x7fc1040665a0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 -- 172.21.15.92:0/4194081000 shutdown_connections 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b738070 0x56511b73a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b737a00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 --2- 172.21.15.92:0/4194081000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 0x56511b732930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 -- 172.21.15.92:0/4194081000 >> 172.21.15.92:0/4194081000 conn(0x56511b729670 msgr2=0x56511b729a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 -- 172.21.15.92:0/4194081000 shutdown_connections 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 -- 172.21.15.92:0/4194081000 wait complete. 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.785+0000 7fc11d26e740 1 -- start start 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc11d26e740 1 Processor -- start 2024-03-19T01:54:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732510 0x56511b73c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 0x56511b73ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 0x56511b743a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc11d26e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56511b73afc0 con 0x56511b732510 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc11d26e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56511b64edb0 con 0x56511b738070 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc11d26e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56511b73b140 con 0x56511b732f00 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc11d26e740 10 client.0 _ll_register_callbacks cb 0x56511b653e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732510 0x56511b73c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732510 0x56511b73c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39640/0 (socket says 172.21.15.92:39640) 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc113fff640 1 -- 172.21.15.92:0/2780230837 learned_addr learned my addr 172.21.15.92:0/2780230837 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc1137fe640 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 0x56511b73ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc118c1e640 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 0x56511b743a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc113fff640 1 -- 172.21.15.92:0/2780230837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 msgr2=0x56511b73ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc113fff640 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 0x56511b73ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc113fff640 1 -- 172.21.15.92:0/2780230837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 msgr2=0x56511b743a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc113fff640 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 0x56511b743a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.786+0000 7fc113fff640 1 -- 172.21.15.92:0/2780230837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56511b64dc60 con 0x56511b732510 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc113fff640 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732510 0x56511b73c4e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc1040cc890 tx=0x7fc10400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a60af1279acc2b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc0f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc0f37fe640 1 -- 172.21.15.92:0/2780230837 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1040af070 con 0x56511b732510 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc0f37fe640 1 -- 172.21.15.92:0/2780230837 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc1040af8c0 con 0x56511b732510 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc11d26e740 1 -- 172.21.15.92:0/2780230837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56511b6a0120 con 0x56511b732510 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc0f37fe640 1 -- 172.21.15.92:0/2780230837 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc10409b320 con 0x56511b732510 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc11d26e740 10 client.15405 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc11d26e740 1 -- 172.21.15.92:0/2780230837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56511b3a7210 con 0x56511b732510 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc0f37fe640 1 -- 172.21.15.92:0/2780230837 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56511b6a0120 con 0x56511b732510 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc0f37fe640 1 -- 172.21.15.92:0/2780230837 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fc1040860a0 con 0x56511b732510 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc11d26e740 10 client.15405 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc11d26e740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc11d26e740 20 client.15405 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc11d26e740 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56511b747f50 0x56511b74a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc11d26e740 4 client.15405 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc11d26e740 1 -- 172.21.15.92:0/2780230837 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56511b69a470 con 0x56511b747f50 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.787+0000 7fc1137fe640 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56511b747f50 0x56511b74a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.788+0000 7fc0f37fe640 10 client.15405 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.788+0000 7fc1137fe640 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56511b747f50 0x56511b74a340 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b171cd8a871bf2b server_cookie=7851e2612e851859 in_seq=0 out_seq=0 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc0f37fe640 1 -- 172.21.15.92:0/2780230837 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56511b69a470 con 0x56511b747f50 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc0f37fe640 10 client.15405 handle_command_reply: tid=0 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 client.15405 shutdown 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 -- 172.21.15.92:0/2780230837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732510 msgr2=0x56511b73c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732510 0x56511b73c4e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc1040cc890 tx=0x7fc10400c7d0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 -- 172.21.15.92:0/2780230837 shutdown_connections 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56511b747f50 0x56511b74a340 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 0x56511b743a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc0f37fe640 0 client.15405 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 0x56511b73ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 --2- 172.21.15.92:0/2780230837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732510 0x56511b73c4e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 -- 172.21.15.92:0/2780230837 >> 172.21.15.92:0/2780230837 conn(0x56511b729670 msgr2=0x56511b72a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 -- 172.21.15.92:0/2780230837 shutdown_connections 2024-03-19T01:54:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 1 -- 172.21.15.92:0/2780230837 wait complete. 2024-03-19T01:54:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.789+0000 7fc11d26e740 20 client.15405 trim_cache size 0 max 16384 2024-03-19T01:54:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.803+0000 7fc11d26e740 1 Processor -- start 2024-03-19T01:54:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.803+0000 7fc11d26e740 1 -- start start 2024-03-19T01:54:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.803+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 0x56511b728220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.803+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 0x56511b743a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.803+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b738070 0x56511b728760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.803+0000 7fc11d26e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141a6a80 con 0x56511b738070 2024-03-19T01:54:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.803+0000 7fc11d26e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141a85f0 con 0x56511b732510 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.803+0000 7fc11d26e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141a7150 con 0x56511b732f00 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc118c1e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b738070 0x56511b728760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc118c1e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b738070 0x56511b728760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39654/0 (socket says 172.21.15.92:39654) 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc118c1e640 1 -- 172.21.15.92:0/597573308 learned_addr learned my addr 172.21.15.92:0/597573308 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc113fff640 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 0x56511b728220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc1137fe640 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 0x56511b743a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc118c1e640 1 -- 172.21.15.92:0/597573308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 msgr2=0x56511b743a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc118c1e640 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 0x56511b743a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc118c1e640 1 -- 172.21.15.92:0/597573308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 msgr2=0x56511b728220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc118c1e640 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 0x56511b728220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc118c1e640 1 -- 172.21.15.92:0/597573308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56511b74a6b0 con 0x56511b738070 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc1137fe640 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 0x56511b743a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:54:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc113fff640 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 0x56511b728220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc118c1e640 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b738070 0x56511b728760 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00f980 tx=0x7fc10c013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e302caec0ca8625 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc0f9ffb640 1 -- 172.21.15.92:0/597573308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c02e070 con 0x56511b738070 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc0f9ffb640 1 -- 172.21.15.92:0/597573308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10c01ee20 con 0x56511b738070 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.804+0000 7fc0f9ffb640 1 -- 172.21.15.92:0/597573308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c01c420 con 0x56511b738070 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 -- 172.21.15.92:0/597573308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b738070 msgr2=0x56511b728760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b738070 0x56511b728760 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00f980 tx=0x7fc10c013030 comp rx=0 tx=0).stop 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 -- 172.21.15.92:0/597573308 shutdown_connections 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b738070 0x56511b728760 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732f00 0x56511b743a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 --2- 172.21.15.92:0/597573308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b732510 0x56511b728220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 -- 172.21.15.92:0/597573308 >> 172.21.15.92:0/597573308 conn(0x56511b729670 msgr2=0x56511b72d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 -- 172.21.15.92:0/597573308 shutdown_connections 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 -- 172.21.15.92:0/597573308 wait complete. 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 -- start start 2024-03-19T01:54:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.805+0000 7fc11d26e740 1 Processor -- start 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732510 0x56511b7418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b741e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc11d26e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 0x56511b742e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc11d26e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141a7ef0 con 0x56511b732f00 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc11d26e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141a7820 con 0x56511b738070 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc11d26e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141a4a60 con 0x56511b732510 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc11d26e740 10 client.0 _ll_register_callbacks cb 0x7fc10c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc1137fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b741e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc113fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732510 0x56511b7418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc1137fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b741e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39664/0 (socket says 172.21.15.92:39664) 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc118c1e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 0x56511b742e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc113fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732510 0x56511b7418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:44822/0 (socket says 172.21.15.92:44822) 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc1137fe640 1 -- 172.21.15.92:0/795510859 learned_addr learned my addr 172.21.15.92:0/795510859 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc1137fe640 1 -- 172.21.15.92:0/795510859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732510 msgr2=0x56511b7418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc1137fe640 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732510 0x56511b7418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc1137fe640 1 -- 172.21.15.92:0/795510859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 msgr2=0x56511b742e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc1137fe640 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 0x56511b742e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc1137fe640 1 -- 172.21.15.92:0/795510859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56511b473620 con 0x56511b732f00 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc118c1e640 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 0x56511b742e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc1137fe640 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b741e30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc10802cfd0 tx=0x7fc10802cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e5c6232ad0dcce server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc0f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc0f37fe640 1 -- 172.21.15.92:0/795510859 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108046ca0 con 0x56511b732f00 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc0f37fe640 1 -- 172.21.15.92:0/795510859 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc108004040 con 0x56511b732f00 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.806+0000 7fc11d26e740 1 -- 172.21.15.92:0/795510859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10c01ee20 con 0x56511b732f00 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc0f37fe640 1 -- 172.21.15.92:0/795510859 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc108055050 con 0x56511b732f00 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc11d26e740 10 client.15411 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc11d26e740 1 -- 172.21.15.92:0/795510859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56511b7ae800 con 0x56511b732f00 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc0f37fe640 1 -- 172.21.15.92:0/795510859 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10c01ee20 con 0x56511b732f00 2024-03-19T01:54:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc0f37fe640 1 -- 172.21.15.92:0/795510859 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fc1080051f0 con 0x56511b732f00 2024-03-19T01:54:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc11d26e740 10 client.15411 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc11d26e740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc11d26e740 20 client.15411 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc11d26e740 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56511b7af2b0 0x56511b7b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc11d26e740 4 client.15411 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:54:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc11d26e740 1 -- 172.21.15.92:0/795510859 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56511b618770 con 0x56511b7af2b0 2024-03-19T01:54:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc113fff640 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56511b7af2b0 0x56511b7b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc0f37fe640 10 client.15411 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.807+0000 7fc113fff640 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56511b7af2b0 0x56511b7b16d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=822db98a072b90f7 server_cookie=b1ea2815c0359c0c in_seq=0 out_seq=0 2024-03-19T01:54:00.815 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:00 smithi092 ceph-mon[62574]: pgmap v387: 97 pgs: 97 active+clean; 781 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc0f37fe640 1 -- 172.21.15.92:0/795510859 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56511b618770 con 0x56511b7af2b0 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc0f37fe640 10 client.15411 handle_command_reply: tid=0 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 client.15411 shutdown 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 -- 172.21.15.92:0/795510859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 msgr2=0x56511b741e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b741e30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc10802cfd0 tx=0x7fc10802cbf0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 -- 172.21.15.92:0/795510859 shutdown_connections 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56511b7af2b0 0x56511b7b16d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56511b738070 0x56511b742e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc0f37fe640 0 client.15411 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56511b732f00 0x56511b741e30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 --2- 172.21.15.92:0/795510859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56511b732510 0x56511b7418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 -- 172.21.15.92:0/795510859 >> 172.21.15.92:0/795510859 conn(0x56511b729670 msgr2=0x56511b733970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 -- 172.21.15.92:0/795510859 shutdown_connections 2024-03-19T01:54:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 1 -- 172.21.15.92:0/795510859 wait complete. 2024-03-19T01:54:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.814+0000 7fc11d26e740 20 client.15411 trim_cache size 0 max 16384 2024-03-19T01:54:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:54:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:54:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:54:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:54:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 -- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc0fc0a2940 msgr2=0x7fc0fc0a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc0fc0a2940 0x7fc0fc0a4e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009fd0 tx=0x7fc10c004a40 comp rx=0 tx=0).stop 2024-03-19T01:54:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 -- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114101850 msgr2=0x7fc114197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114101850 0x7fc114197a80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc10400e9c0 tx=0x7fc10400ee80 comp rx=0 tx=0).stop 2024-03-19T01:54:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 -- 172.21.15.92:0/2459924812 shutdown_connections 2024-03-19T01:54:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc114103400 0x7fc11419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc0fc0a2940 0x7fc0fc0a4e00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc114102a40 0x7fc114197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 --2- 172.21.15.92:0/2459924812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc114101850 0x7fc114197a80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 -- 172.21.15.92:0/2459924812 >> 172.21.15.92:0/2459924812 conn(0x7fc1140fcde0 msgr2=0x7fc114105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 -- 172.21.15.92:0/2459924812 shutdown_connections 2024-03-19T01:54:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:00.815+0000 7fc11a681640 1 -- 172.21.15.92:0/2459924812 wait complete. 2024-03-19T01:54:00.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:00 smithi144 ceph-mon[67199]: pgmap v387: 97 pgs: 97 active+clean; 781 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:54:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:00 smithi159 ceph-mon[67110]: pgmap v387: 97 pgs: 97 active+clean; 781 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:54:02.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:02 smithi144 ceph-mon[67199]: pgmap v388: 97 pgs: 97 active+clean; 781 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T01:54:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:02 smithi159 ceph-mon[67110]: pgmap v388: 97 pgs: 97 active+clean; 781 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T01:54:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:02 smithi092 ceph-mon[62574]: pgmap v388: 97 pgs: 97 active+clean; 781 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T01:54:04.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:04 smithi144 ceph-mon[67199]: pgmap v389: 97 pgs: 97 active+clean; 812 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T01:54:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:04 smithi159 ceph-mon[67110]: pgmap v389: 97 pgs: 97 active+clean; 812 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T01:54:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:04 smithi092 ceph-mon[62574]: pgmap v389: 97 pgs: 97 active+clean; 812 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T01:54:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:06 smithi144 ceph-mon[67199]: pgmap v390: 97 pgs: 97 active+clean; 812 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:54:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:06 smithi159 ceph-mon[67110]: pgmap v390: 97 pgs: 97 active+clean; 812 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:54:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:06 smithi092 ceph-mon[62574]: pgmap v390: 97 pgs: 97 active+clean; 812 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:54:08.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:08 smithi144 ceph-mon[67199]: pgmap v391: 97 pgs: 97 active+clean; 812 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:54:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:08 smithi159 ceph-mon[67110]: pgmap v391: 97 pgs: 97 active+clean; 812 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:54:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:08 smithi092 ceph-mon[62574]: pgmap v391: 97 pgs: 97 active+clean; 812 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:54:10.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:10 smithi144 ceph-mon[67199]: pgmap v392: 97 pgs: 97 active+clean; 829 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:54:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:10 smithi159 ceph-mon[67110]: pgmap v392: 97 pgs: 97 active+clean; 829 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:54:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:10 smithi092 ceph-mon[62574]: pgmap v392: 97 pgs: 97 active+clean; 829 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:54:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:12 smithi144 ceph-mon[67199]: pgmap v393: 97 pgs: 97 active+clean; 829 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:54:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:54:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:12 smithi144 ceph-mon[67199]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-19T01:54:12.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:12.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:12.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:12.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:12 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:12 smithi159 ceph-mon[67110]: pgmap v393: 97 pgs: 97 active+clean; 829 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:54:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:54:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:12 smithi159 ceph-mon[67110]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-19T01:54:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:12.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:12.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:12.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:12 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:12 smithi092 ceph-mon[62574]: pgmap v393: 97 pgs: 97 active+clean; 829 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:54:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:54:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:12 smithi092 ceph-mon[62574]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-19T01:54:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:13.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:13.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:12 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:13.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:54:13.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:54:13.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.906+0000 7fad2b722640 1 -- 172.21.15.92:0/1800913592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 msgr2=0x7fad24101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.906+0000 7fad2b722640 1 --2- 172.21.15.92:0/1800913592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 0x7fad24101dd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fad14009900 tx=0x7fad1402bc30 comp rx=0 tx=0).stop 2024-03-19T01:54:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.906+0000 7fad2b722640 1 -- 172.21.15.92:0/1800913592 shutdown_connections 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.906+0000 7fad2b722640 1 --2- 172.21.15.92:0/1800913592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad24102310 0x7fad24108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.906+0000 7fad2b722640 1 --2- 172.21.15.92:0/1800913592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 0x7fad24101dd0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.906+0000 7fad2b722640 1 --2- 172.21.15.92:0/1800913592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24100760 0x7fad24100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.906+0000 7fad2b722640 1 -- 172.21.15.92:0/1800913592 >> 172.21.15.92:0/1800913592 conn(0x7fad240fbcf0 msgr2=0x7fad240fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 -- 172.21.15.92:0/1800913592 shutdown_connections 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 -- 172.21.15.92:0/1800913592 wait complete. 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 Processor -- start 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 -- start start 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24100760 0x7fad2406abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 0x7fad2419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad24102310 0x7fad241a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad2410c3e0 con 0x7fad24101950 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad2410c260 con 0x7fad24100760 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad2b722640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad2410c560 con 0x7fad24102310 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad28cbd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 0x7fad2419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad28cbd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 0x7fad2419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38696/0 (socket says 172.21.15.92:38696) 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad28cbd640 1 -- 172.21.15.92:0/275150032 learned_addr learned my addr 172.21.15.92:0/275150032 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad29cbf640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad24102310 0x7fad241a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.907+0000 7fad294be640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24100760 0x7fad2406abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad28cbd640 1 -- 172.21.15.92:0/275150032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad24102310 msgr2=0x7fad241a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad28cbd640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad24102310 0x7fad241a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad28cbd640 1 -- 172.21.15.92:0/275150032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24100760 msgr2=0x7fad2406abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad28cbd640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24100760 0x7fad2406abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad28cbd640 1 -- 172.21.15.92:0/275150032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad241a19b0 con 0x7fad24101950 2024-03-19T01:54:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad28cbd640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 0x7fad2419ec70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fad14009230 tx=0x7fad1402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad1a7fc640 1 -- 172.21.15.92:0/275150032 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14041070 con 0x7fad24101950 2024-03-19T01:54:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad1a7fc640 1 -- 172.21.15.92:0/275150032 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad14004450 con 0x7fad24101950 2024-03-19T01:54:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad2b722640 1 -- 172.21.15.92:0/275150032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad241a1be0 con 0x7fad24101950 2024-03-19T01:54:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad1a7fc640 1 -- 172.21.15.92:0/275150032 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14003e20 con 0x7fad24101950 2024-03-19T01:54:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.908+0000 7fad2b722640 1 -- 172.21.15.92:0/275150032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad241a20d0 con 0x7fad24101950 2024-03-19T01:54:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.910+0000 7fad1a7fc640 1 -- 172.21.15.92:0/275150032 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fad14044070 con 0x7fad24101950 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad2e3ed740 1 Processor -- start 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad2e3ed740 1 -- start start 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012100 0x562f48012520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012af0 0x562f480175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f4801a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562f47f76be0 con 0x562f48017c60 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad1a7fc640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad100a28d0 0x7fad100a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f47f1f960 con 0x562f48012af0 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562f47f72c80 con 0x562f48012100 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad29cbf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012100 0x562f48012520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad28cbd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f4801a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad29cbf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012100 0x562f48012520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:36858/0 (socket says 172.21.15.92:36858) 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad29cbf640 1 -- 172.21.15.92:0/3887084155 learned_addr learned my addr 172.21.15.92:0/3887084155 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad294be640 1 --2- 172.21.15.92:0/3887084155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012af0 0x562f480175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad29cbf640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad100a28d0 0x7fad100a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad1a7fc640 1 -- 172.21.15.92:0/275150032 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fad140f0550 con 0x7fad24101950 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad29cbf640 1 -- 172.21.15.92:0/3887084155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012af0 msgr2=0x562f480175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad29cbf640 1 --2- 172.21.15.92:0/3887084155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012af0 0x562f480175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad29cbf640 1 -- 172.21.15.92:0/3887084155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 msgr2=0x562f4801a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad29cbf640 1 --2- 172.21.15.92:0/3887084155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f4801a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.911+0000 7fad29cbf640 1 -- 172.21.15.92:0/3887084155 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f47f16d20 con 0x562f48012100 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.912+0000 7fad29cbf640 1 --2- 172.21.15.92:0/3887084155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012100 0x562f48012520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad1c011a30 tx=0x7fad1c011ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7a9665b465b3288 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.912+0000 7fad29cbf640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad100a28d0 0x7fad100a4d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00f850 tx=0x7fad1c018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.912+0000 7fad0f7fe640 1 -- 172.21.15.92:0/3887084155 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c004030 con 0x562f48012100 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.912+0000 7fad0f7fe640 1 -- 172.21.15.92:0/3887084155 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad1c009a20 con 0x562f48012100 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.912+0000 7fad0f7fe640 1 -- 172.21.15.92:0/3887084155 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c003a30 con 0x562f48012100 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.912+0000 7fad2e3ed740 1 -- 172.21.15.92:0/3887084155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012100 msgr2=0x562f48012520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.912+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/3887084155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012100 0x562f48012520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad1c011a30 tx=0x7fad1c011ef0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 -- 172.21.15.92:0/3887084155 shutdown_connections 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/3887084155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f4801a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/3887084155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012af0 0x562f480175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/3887084155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012100 0x562f48012520 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 -- 172.21.15.92:0/3887084155 >> 172.21.15.92:0/3887084155 conn(0x562f4800d670 msgr2=0x562f4800da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 -- 172.21.15.92:0/3887084155 shutdown_connections 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 -- 172.21.15.92:0/3887084155 wait complete. 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 -- start start 2024-03-19T01:54:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 Processor -- start 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 0x562f4801c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f4801c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48023630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562f4801ab50 con 0x562f48017c60 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f47f32db0 con 0x562f48012100 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562f4801acd0 con 0x562f48012af0 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad2e3ed740 10 client.0 _ll_register_callbacks cb 0x562f47f37e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.913+0000 7fad29cbf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48023630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad28cbd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f4801c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad29cbf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48023630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38714/0 (socket says 172.21.15.92:38714) 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad28cbd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f4801c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:36874/0 (socket says 172.21.15.92:36874) 2024-03-19T01:54:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad29cbf640 1 -- 172.21.15.92:0/1977552980 learned_addr learned my addr 172.21.15.92:0/1977552980 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad294be640 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 0x562f4801c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad29cbf640 1 -- 172.21.15.92:0/1977552980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 msgr2=0x562f4801c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad29cbf640 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f4801c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad29cbf640 1 -- 172.21.15.92:0/1977552980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 msgr2=0x562f4801c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad29cbf640 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 0x562f4801c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad29cbf640 1 -- 172.21.15.92:0/1977552980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f47f31c60 con 0x562f48017c60 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad29cbf640 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48023630 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fad1c000c00 tx=0x7fad1c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e61d18536b0f7f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad0cff9640 1 -- 172.21.15.92:0/1977552980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c034060 con 0x562f48017c60 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad2e3ed740 1 -- 172.21.15.92:0/1977552980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad1c009a20 con 0x562f48017c60 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad0cff9640 1 -- 172.21.15.92:0/1977552980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fad1c011560 con 0x562f48017c60 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad0cff9640 1 -- 172.21.15.92:0/1977552980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad1c005260 con 0x562f48017c60 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad2e3ed740 10 client.15423 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad2e3ed740 1 -- 172.21.15.92:0/1977552980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f47c8b210 con 0x562f48017c60 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.914+0000 7fad0cff9640 1 -- 172.21.15.92:0/1977552980 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad1c009a20 con 0x562f48017c60 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad0cff9640 1 -- 172.21.15.92:0/1977552980 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fad1c024030 con 0x562f48017c60 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad2e3ed740 10 client.15423 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad2e3ed740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad2e3ed740 20 client.15423 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562f48027e30 0x562f4802a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad2e3ed740 4 client.15423 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad2e3ed740 1 -- 172.21.15.92:0/1977552980 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f47f84120 con 0x562f48027e30 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad294be640 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562f48027e30 0x562f4802a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad0cff9640 10 client.15423 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.915+0000 7fad294be640 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562f48027e30 0x562f4802a220 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8465ceda5e8258b server_cookie=9af65b72dc04b2bd in_seq=0 out_seq=0 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad0cff9640 1 -- 172.21.15.92:0/1977552980 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562f47f84120 con 0x562f48027e30 2024-03-19T01:54:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad0cff9640 10 client.15423 handle_command_reply: tid=0 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 client.15423 shutdown 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 -- 172.21.15.92:0/1977552980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 msgr2=0x562f48023630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48023630 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fad1c000c00 tx=0x7fad1c00ed50 comp rx=0 tx=0).stop 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 -- 172.21.15.92:0/1977552980 shutdown_connections 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562f48027e30 0x562f4802a220 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48023630 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad0cff9640 0 client.15423 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f4801c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/1977552980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 0x562f4801c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 -- 172.21.15.92:0/1977552980 >> 172.21.15.92:0/1977552980 conn(0x562f4800d670 msgr2=0x562f4800e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 -- 172.21.15.92:0/1977552980 shutdown_connections 2024-03-19T01:54:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 1 -- 172.21.15.92:0/1977552980 wait complete. 2024-03-19T01:54:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.916+0000 7fad2e3ed740 20 client.15423 trim_cache size 0 max 16384 2024-03-19T01:54:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad2e3ed740 1 Processor -- start 2024-03-19T01:54:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad2e3ed740 1 -- start start 2024-03-19T01:54:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 0x562f4800cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f480235f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48010fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad241aaeb0 con 0x562f48017c60 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad241aca20 con 0x562f48012100 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad241ab580 con 0x562f48012af0 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad29cbf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48010fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad29cbf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48010fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38728/0 (socket says 172.21.15.92:38728) 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad29cbf640 1 -- 172.21.15.92:0/165390967 learned_addr learned my addr 172.21.15.92:0/165390967 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad28cbd640 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f480235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad294be640 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 0x562f4800cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad29cbf640 1 -- 172.21.15.92:0/165390967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 msgr2=0x562f480235f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad29cbf640 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f480235f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad29cbf640 1 -- 172.21.15.92:0/165390967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 msgr2=0x562f4800cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad29cbf640 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 0x562f4800cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad29cbf640 1 -- 172.21.15.92:0/165390967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f47d57620 con 0x562f48017c60 2024-03-19T01:54:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad28cbd640 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f480235f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad294be640 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 0x562f4800cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad29cbf640 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48010fe0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fad1c01e400 tx=0x7fad1c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca9feae067a7d9a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad0effd640 1 -- 172.21.15.92:0/165390967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c03f030 con 0x562f48017c60 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad0effd640 1 -- 172.21.15.92:0/165390967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad1c00edc0 con 0x562f48017c60 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.931+0000 7fad0effd640 1 -- 172.21.15.92:0/165390967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c034070 con 0x562f48017c60 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 -- 172.21.15.92:0/165390967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 msgr2=0x562f48010fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48010fe0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fad1c01e400 tx=0x7fad1c019160 comp rx=0 tx=0).stop 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 -- 172.21.15.92:0/165390967 shutdown_connections 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48017c60 0x562f48010fe0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f480235f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/165390967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48012100 0x562f4800cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 -- 172.21.15.92:0/165390967 >> 172.21.15.92:0/165390967 conn(0x562f4800d670 msgr2=0x562f480108d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 -- 172.21.15.92:0/165390967 shutdown_connections 2024-03-19T01:54:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 -- 172.21.15.92:0/165390967 wait complete. 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.932+0000 7fad2e3ed740 1 -- start start 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad2e3ed740 1 Processor -- start 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48012100 0x562f480215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f48021b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad2e3ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48017c60 0x562f48022aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad241ac320 con 0x562f48012100 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad241abc50 con 0x562f48017c60 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad2e3ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad241a8e90 con 0x562f48012af0 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad2e3ed740 10 client.0 _ll_register_callbacks cb 0x7fad14030f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad294be640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48012100 0x562f480215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad294be640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48012100 0x562f480215c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38742/0 (socket says 172.21.15.92:38742) 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad294be640 1 -- 172.21.15.92:0/231359603 learned_addr learned my addr 172.21.15.92:0/231359603 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad28cbd640 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f48021b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad29cbf640 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48017c60 0x562f48022aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad294be640 1 -- 172.21.15.92:0/231359603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 msgr2=0x562f48021b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad294be640 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f48021b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad294be640 1 -- 172.21.15.92:0/231359603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48017c60 msgr2=0x562f48022aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad294be640 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48017c60 0x562f48022aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.933+0000 7fad294be640 1 -- 172.21.15.92:0/231359603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f48023090 con 0x562f48012100 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad294be640 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48012100 0x562f480215c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x562f48013560 tx=0x7fad20003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ad6079c67db0c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad0cff9640 1 -- 172.21.15.92:0/231359603 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20022070 con 0x562f48012100 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad0cff9640 1 -- 172.21.15.92:0/231359603 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fad2001eba0 con 0x562f48012100 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad2e3ed740 1 -- 172.21.15.92:0/231359603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad1c00edc0 con 0x562f48012100 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad0cff9640 1 -- 172.21.15.92:0/231359603 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad2002e050 con 0x562f48012100 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad2e3ed740 10 client.15429 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad2e3ed740 1 -- 172.21.15.92:0/231359603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f4808ea80 con 0x562f48012100 2024-03-19T01:54:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad0cff9640 1 -- 172.21.15.92:0/231359603 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad1c00edc0 con 0x562f48012100 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad0cff9640 1 -- 172.21.15.92:0/231359603 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fad200043d0 con 0x562f48012100 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad2e3ed740 10 client.15429 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad2e3ed740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad2e3ed740 20 client.15429 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562f4808f490 0x562f480918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad2e3ed740 4 client.15429 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.934+0000 7fad2e3ed740 1 -- 172.21.15.92:0/231359603 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f47f7e470 con 0x562f4808f490 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.935+0000 7fad28cbd640 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562f4808f490 0x562f480918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.935+0000 7fad0cff9640 10 client.15429 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.935+0000 7fad28cbd640 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562f4808f490 0x562f480918b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23f4fb0ec176031 server_cookie=a713c77f6ad0a1ed in_seq=0 out_seq=0 2024-03-19T01:54:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad0cff9640 1 -- 172.21.15.92:0/231359603 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f47f7e470 con 0x562f4808f490 2024-03-19T01:54:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad0cff9640 10 client.15429 handle_command_reply: tid=0 2024-03-19T01:54:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad2e3ed740 1 client.15429 shutdown 2024-03-19T01:54:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad2e3ed740 1 -- 172.21.15.92:0/231359603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48012100 msgr2=0x562f480215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48012100 0x562f480215c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x562f48013560 tx=0x7fad20003d40 comp rx=0 tx=0).stop 2024-03-19T01:54:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad2e3ed740 1 -- 172.21.15.92:0/231359603 shutdown_connections 2024-03-19T01:54:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562f4808f490 0x562f480918b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f48017c60 0x562f48022aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f48012af0 0x562f48021b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad0cff9640 0 client.15429 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.941+0000 7fad2e3ed740 1 --2- 172.21.15.92:0/231359603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562f48012100 0x562f480215c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2e3ed740 1 -- 172.21.15.92:0/231359603 >> 172.21.15.92:0/231359603 conn(0x562f4800d670 msgr2=0x562f48011160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2e3ed740 1 -- 172.21.15.92:0/231359603 shutdown_connections 2024-03-19T01:54:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2e3ed740 1 -- 172.21.15.92:0/231359603 wait complete. 2024-03-19T01:54:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2e3ed740 20 client.15429 trim_cache size 0 max 16384 2024-03-19T01:54:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:54:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:54:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:54:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:54:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 -- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad100a28d0 msgr2=0x7fad100a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad100a28d0 0x7fad100a4d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00f850 tx=0x7fad1c018000 comp rx=0 tx=0).stop 2024-03-19T01:54:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 -- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 msgr2=0x7fad2419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 0x7fad2419ec70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fad14009230 tx=0x7fad1402c650 comp rx=0 tx=0).stop 2024-03-19T01:54:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 -- 172.21.15.92:0/275150032 shutdown_connections 2024-03-19T01:54:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fad100a28d0 0x7fad100a4d90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad24102310 0x7fad241a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad24101950 0x7fad2419ec70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 --2- 172.21.15.92:0/275150032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24100760 0x7fad2406abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 -- 172.21.15.92:0/275150032 >> 172.21.15.92:0/275150032 conn(0x7fad240fbcf0 msgr2=0x7fad24104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 -- 172.21.15.92:0/275150032 shutdown_connections 2024-03-19T01:54:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:13.942+0000 7fad2b722640 1 -- 172.21.15.92:0/275150032 wait complete. 2024-03-19T01:54:13.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:54:13.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:54:13.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:14.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:54:14.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:54:14.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:54:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:14 smithi144 ceph-mon[67199]: pgmap v394: 97 pgs: 97 active+clean; 855 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:54:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:14 smithi159 ceph-mon[67110]: pgmap v394: 97 pgs: 97 active+clean; 855 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:54:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:14 smithi092 ceph-mon[62574]: pgmap v394: 97 pgs: 97 active+clean; 855 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:54:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:16 smithi144 ceph-mon[67199]: pgmap v395: 97 pgs: 97 active+clean; 855 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:54:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:16 smithi159 ceph-mon[67110]: pgmap v395: 97 pgs: 97 active+clean; 855 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:54:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:16 smithi092 ceph-mon[62574]: pgmap v395: 97 pgs: 97 active+clean; 855 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:54:18.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:18 smithi144 ceph-mon[67199]: pgmap v396: 97 pgs: 97 active+clean; 855 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T01:54:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:18 smithi159 ceph-mon[67110]: pgmap v396: 97 pgs: 97 active+clean; 855 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T01:54:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:18 smithi092 ceph-mon[62574]: pgmap v396: 97 pgs: 97 active+clean; 855 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T01:54:20.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:20 smithi144 ceph-mon[67199]: pgmap v397: 97 pgs: 97 active+clean; 879 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:54:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:20 smithi159 ceph-mon[67110]: pgmap v397: 97 pgs: 97 active+clean; 879 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:54:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:20 smithi092 ceph-mon[62574]: pgmap v397: 97 pgs: 97 active+clean; 879 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:54:22.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:22 smithi144 ceph-mon[67199]: pgmap v398: 97 pgs: 97 active+clean; 879 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2024-03-19T01:54:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:22 smithi159 ceph-mon[67110]: pgmap v398: 97 pgs: 97 active+clean; 879 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2024-03-19T01:54:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:22 smithi092 ceph-mon[62574]: pgmap v398: 97 pgs: 97 active+clean; 879 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 16 op/s 2024-03-19T01:54:24.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:24 smithi144 ceph-mon[67199]: pgmap v399: 97 pgs: 97 active+clean; 904 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T01:54:24.862 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:24 smithi092 ceph-mon[62574]: pgmap v399: 97 pgs: 97 active+clean; 904 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T01:54:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:24 smithi159 ceph-mon[67110]: pgmap v399: 97 pgs: 97 active+clean; 904 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T01:54:26.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:26 smithi144 ceph-mon[67199]: pgmap v400: 97 pgs: 97 active+clean; 904 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:54:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:26 smithi159 ceph-mon[67110]: pgmap v400: 97 pgs: 97 active+clean; 904 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:54:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.029+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/207042142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8101870 msgr2=0x7ffbd8101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.029+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/207042142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8101870 0x7ffbd8101c90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc802bc30 comp rx=0 tx=0).stop 2024-03-19T01:54:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.030+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/207042142 shutdown_connections 2024-03-19T01:54:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.030+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/207042142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbd8103420 0x7ffbd8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.030+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/207042142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd8102a60 0x7ffbd8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.030+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/207042142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8101870 0x7ffbd8101c90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.030+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/207042142 >> 172.21.15.92:0/207042142 conn(0x7ffbd80fcde0 msgr2=0x7ffbd80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.030+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/207042142 shutdown_connections 2024-03-19T01:54:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.030+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/207042142 wait complete. 2024-03-19T01:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.030+0000 7ffbdc8b2640 1 Processor -- start 2024-03-19T01:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbdc8b2640 1 -- start start 2024-03-19T01:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbdc8b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd8101870 0x7ffbd8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbdc8b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8102a60 0x7ffbd8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbdc8b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbd8103420 0x7ffbd819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbdc8b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd810d4f0 con 0x7ffbd8102a60 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbdc8b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd810d370 con 0x7ffbd8101870 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbdc8b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd810d670 con 0x7ffbd8103420 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8102a60 0x7ffbd8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8102a60 0x7ffbd8197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56714/0 (socket says 172.21.15.92:56714) 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd8101870 0x7ffbd8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1879890131 learned_addr learned my addr 172.21.15.92:0/1879890131 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd6d9d640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbd8103420 0x7ffbd819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1879890131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbd8103420 msgr2=0x7ffbd819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbd8103420 0x7ffbd819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1879890131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd8101870 msgr2=0x7ffbd8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd8101870 0x7ffbd8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1879890131 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd819f6d0 con 0x7ffbd8102a60 2024-03-19T01:54:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8102a60 0x7ffbd8197ff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00e9f0 tx=0x7ffbcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.031+0000 7ffbc77fe640 1 -- 172.21.15.92:0/1879890131 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00cd80 con 0x7ffbd8102a60 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.032+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/1879890131 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd819f9c0 con 0x7ffbd8102a60 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.032+0000 7ffbc77fe640 1 -- 172.21.15.92:0/1879890131 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc010910 con 0x7ffbd8102a60 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.032+0000 7ffbc77fe640 1 -- 172.21.15.92:0/1879890131 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0188f0 con 0x7ffbd8102a60 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.032+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/1879890131 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd819ff60 con 0x7ffbd8102a60 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.034+0000 7ffbc77fe640 1 -- 172.21.15.92:0/1879890131 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ffbcc010ac0 con 0x7ffbd8102a60 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.034+0000 7ffbc77fe640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffbc00a2920 0x7ffbc00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.034+0000 7ffbdf57f740 1 Processor -- start 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.034+0000 7ffbd659c640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffbc00a2920 0x7ffbc00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.034+0000 7ffbc77fe640 1 -- 172.21.15.92:0/1879890131 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ffbcc0cee60 con 0x7ffbd8102a60 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.034+0000 7ffbdf57f740 1 -- start start 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.034+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 0x55fcb3289520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.034+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289af0 0x55fcb328e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb32910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fcb31edbe0 con 0x55fcb328ec60 2024-03-19T01:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fcb3196960 con 0x55fcb3289100 2024-03-19T01:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fcb31e9c80 con 0x55fcb3289af0 2024-03-19T01:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb32910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd659c640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffbc00a2920 0x7ffbc00a4de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd5d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289af0 0x55fcb328e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb32910f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56716/0 (socket says 172.21.15.92:56716) 2024-03-19T01:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd6d9d640 1 -- 172.21.15.92:0/1634336103 learned_addr learned my addr 172.21.15.92:0/1634336103 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd659c640 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 0x55fcb3289520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1634336103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 msgr2=0x55fcb3289520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 0x55fcb3289520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1634336103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 msgr2=0x55fcb32910f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb32910f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1634336103 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcb318dd20 con 0x55fcb3289af0 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd6d9d640 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb32910f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd659c640 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 0x55fcb3289520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289af0 0x55fcb328e5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00e9c0 tx=0x7ffbcc09b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37a67b16c71d1608 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbc4ff9640 1 -- 172.21.15.92:0/1634336103 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc099b60 con 0x55fcb3289af0 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbc4ff9640 1 -- 172.21.15.92:0/1634336103 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc0cca20 con 0x55fcb3289af0 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.035+0000 7ffbc4ff9640 1 -- 172.21.15.92:0/1634336103 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc09af10 con 0x55fcb3289af0 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.036+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1634336103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289af0 msgr2=0x55fcb328e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.036+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289af0 0x55fcb328e5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00e9c0 tx=0x7ffbcc09b920 comp rx=0 tx=0).stop 2024-03-19T01:54:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.036+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1634336103 shutdown_connections 2024-03-19T01:54:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.036+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb32910f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.036+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289af0 0x55fcb328e5f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.036+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1634336103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 0x55fcb3289520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.036+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1634336103 >> 172.21.15.92:0/1634336103 conn(0x55fcb3284670 msgr2=0x55fcb3284a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1634336103 shutdown_connections 2024-03-19T01:54:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1634336103 wait complete. 2024-03-19T01:54:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 -- start start 2024-03-19T01:54:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 Processor -- start 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289100 0x55fcb3293280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 0x55fcb32937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb329a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fcb3291c10 con 0x55fcb328ec60 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fcb31a9db0 con 0x55fcb3289af0 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fcb3291d90 con 0x55fcb3289100 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbdf57f740 10 client.0 _ll_register_callbacks cb 0x55fcb31aee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbd6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb329a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbd659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289100 0x55fcb3293280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbd659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289100 0x55fcb3293280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:53954/0 (socket says 172.21.15.92:53954) 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbd6d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb329a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56732/0 (socket says 172.21.15.92:56732) 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbd659c640 1 -- 172.21.15.92:0/3010078499 learned_addr learned my addr 172.21.15.92:0/3010078499 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.037+0000 7ffbd5d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 0x55fcb32937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbd6d9d640 1 -- 172.21.15.92:0/3010078499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289100 msgr2=0x55fcb3293280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbd6d9d640 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289100 0x55fcb3293280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbd6d9d640 1 -- 172.21.15.92:0/3010078499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 msgr2=0x55fcb32937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbd6d9d640 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 0x55fcb32937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbd6d9d640 1 -- 172.21.15.92:0/3010078499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcb31a8c60 con 0x55fcb328ec60 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbd6d9d640 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb329a800 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55fcb32855e0 tx=0x7ffbd000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab131dcdbbc98292 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbbdffb640 1 -- 172.21.15.92:0/3010078499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0009e30 con 0x55fcb328ec60 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbbdffb640 1 -- 172.21.15.92:0/3010078499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ffbd0003970 con 0x55fcb328ec60 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3010078499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbcc0cca20 con 0x55fcb328ec60 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbbdffb640 1 -- 172.21.15.92:0/3010078499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbd0003b70 con 0x55fcb328ec60 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbdf57f740 10 client.15441 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3010078499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcb2f02210 con 0x55fcb328ec60 2024-03-19T01:54:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbbdffb640 1 -- 172.21.15.92:0/3010078499 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc0cca20 con 0x55fcb328ec60 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbbdffb640 1 -- 172.21.15.92:0/3010078499 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ffbd0025450 con 0x55fcb328ec60 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbdf57f740 10 client.15441 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.038+0000 7ffbdf57f740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.039+0000 7ffbdf57f740 20 client.15441 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.039+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fcb329f170 0x55fcb32a1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.039+0000 7ffbdf57f740 4 client.15441 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.039+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3010078499 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcb31fb120 con 0x55fcb329f170 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.039+0000 7ffbd659c640 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fcb329f170 0x55fcb32a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.039+0000 7ffbbdffb640 10 client.15441 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.039+0000 7ffbd659c640 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fcb329f170 0x55fcb32a1560 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9341cfd1be60eb4c server_cookie=970f8dfba02923b1 in_seq=0 out_seq=0 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbbdffb640 1 -- 172.21.15.92:0/3010078499 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fcb31fb120 con 0x55fcb329f170 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbbdffb640 10 client.15441 handle_command_reply: tid=0 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 client.15441 shutdown 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3010078499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 msgr2=0x55fcb329a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb329a800 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55fcb32855e0 tx=0x7ffbd000f790 comp rx=0 tx=0).stop 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3010078499 shutdown_connections 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fcb329f170 0x55fcb32a1560 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb328ec60 0x55fcb329a800 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 0x55fcb32937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbbdffb640 0 client.15441 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3010078499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb3289100 0x55fcb3293280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3010078499 >> 172.21.15.92:0/3010078499 conn(0x55fcb3284670 msgr2=0x55fcb32858c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3010078499 shutdown_connections 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3010078499 wait complete. 2024-03-19T01:54:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.040+0000 7ffbdf57f740 20 client.15441 trim_cache size 0 max 16384 2024-03-19T01:54:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:26 smithi092 ceph-mon[62574]: pgmap v400: 97 pgs: 97 active+clean; 904 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:54:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.061+0000 7ffbdf57f740 1 Processor -- start 2024-03-19T01:54:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.061+0000 7ffbdf57f740 1 -- start start 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.061+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289100 0x55fcb3283930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.061+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 0x55fcb329a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.061+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 0x55fcb328a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.061+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81a8d00 con 0x55fcb3289100 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.061+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81aa870 con 0x55fcb3289af0 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.061+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81a93d0 con 0x55fcb328ec60 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289100 0x55fcb3283930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289100 0x55fcb3283930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56746/0 (socket says 172.21.15.92:56746) 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd659c640 1 -- 172.21.15.92:0/3774371004 learned_addr learned my addr 172.21.15.92:0/3774371004 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd6d9d640 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 0x55fcb328a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 0x55fcb329a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd659c640 1 -- 172.21.15.92:0/3774371004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 msgr2=0x55fcb328a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd659c640 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 0x55fcb328a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd659c640 1 -- 172.21.15.92:0/3774371004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 msgr2=0x55fcb329a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd659c640 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 0x55fcb329a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd659c640 1 -- 172.21.15.92:0/3774371004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcb2fce620 con 0x55fcb3289100 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd6d9d640 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 0x55fcb328a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 0x55fcb329a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbd659c640 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289100 0x55fcb3283930 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffbc802c7b0 tx=0x7ffbc803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa57605713baf73 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbbffff640 1 -- 172.21.15.92:0/3774371004 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8054620 con 0x55fcb3289100 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbbffff640 1 -- 172.21.15.92:0/3774371004 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc80547c0 con 0x55fcb3289100 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.062+0000 7ffbbffff640 1 -- 172.21.15.92:0/3774371004 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8004df0 con 0x55fcb3289100 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3774371004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289100 msgr2=0x55fcb3283930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289100 0x55fcb3283930 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffbc802c7b0 tx=0x7ffbc803e000 comp rx=0 tx=0).stop 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3774371004 shutdown_connections 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 0x55fcb328a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289af0 0x55fcb329a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/3774371004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289100 0x55fcb3283930 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3774371004 >> 172.21.15.92:0/3774371004 conn(0x55fcb3284670 msgr2=0x55fcb3283220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3774371004 shutdown_connections 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 -- 172.21.15.92:0/3774371004 wait complete. 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 -- start start 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 Processor -- start 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 0x55fcb3297ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289af0 0x55fcb32983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 0x55fcb32993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81aa170 con 0x55fcb3289af0 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81a9aa0 con 0x55fcb3289100 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81a6ce0 con 0x55fcb328ec60 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbdf57f740 10 client.0 _ll_register_callbacks cb 0x7ffbd0009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.063+0000 7ffbd5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289af0 0x55fcb32983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289af0 0x55fcb32983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56760/0 (socket says 172.21.15.92:56760) 2024-03-19T01:54:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 0x55fcb3297ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1108463359 learned_addr learned my addr 172.21.15.92:0/1108463359 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd6d9d640 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 0x55fcb32993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1108463359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 msgr2=0x55fcb32993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 0x55fcb32993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1108463359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 msgr2=0x55fcb3297ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 0x55fcb3297ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd5d9b640 1 -- 172.21.15.92:0/1108463359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcb32999e0 con 0x55fcb3289af0 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbd5d9b640 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289af0 0x55fcb32983e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c6a0 tx=0x7ffbcc0a40d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba859edc4b1446b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbbdffb640 1 -- 172.21.15.92:0/1108463359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b3070 con 0x55fcb3289af0 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbbdffb640 1 -- 172.21.15.92:0/1108463359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ffbcc0afb90 con 0x55fcb3289af0 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1108463359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc80547c0 con 0x55fcb3289af0 2024-03-19T01:54:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbbdffb640 1 -- 172.21.15.92:0/1108463359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbcc0862b0 con 0x55fcb3289af0 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbdf57f740 10 client.15447 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1108463359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcb3305450 con 0x55fcb3289af0 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.064+0000 7ffbbdffb640 1 -- 172.21.15.92:0/1108463359 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc80547c0 con 0x55fcb3289af0 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbbdffb640 1 -- 172.21.15.92:0/1108463359 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ffbcc09cae0 con 0x55fcb3289af0 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbdf57f740 10 client.15447 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbdf57f740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbdf57f740 20 client.15447 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fcb3305940 0x55fcb3307d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbdf57f740 4 client.15447 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1108463359 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcb31f5470 con 0x55fcb3305940 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbd659c640 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fcb3305940 0x55fcb3307d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbbdffb640 10 client.15447 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.065+0000 7ffbd659c640 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fcb3305940 0x55fcb3307d60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160e69f6c0645ead server_cookie=d3041fda9b816d99 in_seq=0 out_seq=0 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbbdffb640 1 -- 172.21.15.92:0/1108463359 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fcb31f5470 con 0x55fcb3305940 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbbdffb640 10 client.15447 handle_command_reply: tid=0 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 client.15447 shutdown 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1108463359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289af0 msgr2=0x55fcb32983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289af0 0x55fcb32983e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c6a0 tx=0x7ffbcc0a40d0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1108463359 shutdown_connections 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fcb3305940 0x55fcb3307d60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fcb328ec60 0x55fcb32993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fcb3289af0 0x55fcb32983e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 --2- 172.21.15.92:0/1108463359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fcb3289100 0x55fcb3297ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbbdffb640 0 client.15447 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1108463359 >> 172.21.15.92:0/1108463359 conn(0x55fcb3284670 msgr2=0x55fcb32878d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1108463359 shutdown_connections 2024-03-19T01:54:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 1 -- 172.21.15.92:0/1108463359 wait complete. 2024-03-19T01:54:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.071+0000 7ffbdf57f740 20 client.15447 trim_cache size 0 max 16384 2024-03-19T01:54:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:54:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:54:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:54:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:54:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffbc00a2920 msgr2=0x7ffbc00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffbc00a2920 0x7ffbc00a4de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc8005fd0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8102a60 msgr2=0x7ffbd8197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8102a60 0x7ffbd8197ff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00e9f0 tx=0x7ffbcc00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/1879890131 shutdown_connections 2024-03-19T01:54:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffbc00a2920 0x7ffbc00a4de0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbd8103420 0x7ffbd819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbd8102a60 0x7ffbd8197ff0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 --2- 172.21.15.92:0/1879890131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbd8101870 0x7ffbd8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/1879890131 >> 172.21.15.92:0/1879890131 conn(0x7ffbd80fcde0 msgr2=0x7ffbd8105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/1879890131 shutdown_connections 2024-03-19T01:54:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:27.072+0000 7ffbdc8b2640 1 -- 172.21.15.92:0/1879890131 wait complete. 2024-03-19T01:54:28.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:28 smithi144 ceph-mon[67199]: pgmap v401: 97 pgs: 97 active+clean; 904 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:54:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:28 smithi159 ceph-mon[67110]: pgmap v401: 97 pgs: 97 active+clean; 904 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:54:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:28 smithi092 ceph-mon[62574]: pgmap v401: 97 pgs: 97 active+clean; 904 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:54:30.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:30 smithi144 ceph-mon[67199]: pgmap v402: 97 pgs: 97 active+clean; 929 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-19T01:54:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:30 smithi159 ceph-mon[67110]: pgmap v402: 97 pgs: 97 active+clean; 929 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-19T01:54:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:30 smithi092 ceph-mon[62574]: pgmap v402: 97 pgs: 97 active+clean; 929 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-19T01:54:32.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:32 smithi144 ceph-mon[67199]: pgmap v403: 97 pgs: 97 active+clean; 929 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T01:54:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:32 smithi159 ceph-mon[67110]: pgmap v403: 97 pgs: 97 active+clean; 929 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T01:54:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:32 smithi092 ceph-mon[62574]: pgmap v403: 97 pgs: 97 active+clean; 929 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T01:54:34.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:34 smithi144 ceph-mon[67199]: pgmap v404: 97 pgs: 97 active+clean; 954 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:54:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:34 smithi159 ceph-mon[67110]: pgmap v404: 97 pgs: 97 active+clean; 954 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:54:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:34 smithi092 ceph-mon[62574]: pgmap v404: 97 pgs: 97 active+clean; 954 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:54:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:36 smithi159 ceph-mon[67110]: pgmap v405: 97 pgs: 97 active+clean; 954 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:54:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:36 smithi092 ceph-mon[62574]: pgmap v405: 97 pgs: 97 active+clean; 954 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:54:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:36 smithi144 ceph-mon[67199]: pgmap v405: 97 pgs: 97 active+clean; 954 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:54:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:38 smithi159 ceph-mon[67110]: pgmap v406: 97 pgs: 97 active+clean; 954 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:54:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:38 smithi092 ceph-mon[62574]: pgmap v406: 97 pgs: 97 active+clean; 954 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:54:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:38 smithi144 ceph-mon[67199]: pgmap v406: 97 pgs: 97 active+clean; 954 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:54:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.158+0000 7f24262c6640 1 -- 172.21.15.92:0/394277120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 msgr2=0x7f2420101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.158+0000 7f24262c6640 1 --2- 172.21.15.92:0/394277120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 0x7f2420101db0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2414009900 tx=0x7f241402bc30 comp rx=0 tx=0).stop 2024-03-19T01:54:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.158+0000 7f24262c6640 1 -- 172.21.15.92:0/394277120 shutdown_connections 2024-03-19T01:54:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.158+0000 7f24262c6640 1 --2- 172.21.15.92:0/394277120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24201022f0 0x7f2420108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.158+0000 7f24262c6640 1 --2- 172.21.15.92:0/394277120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 0x7f2420101db0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.158+0000 7f24262c6640 1 --2- 172.21.15.92:0/394277120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2420100740 0x7f2420100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.158+0000 7f24262c6640 1 -- 172.21.15.92:0/394277120 >> 172.21.15.92:0/394277120 conn(0x7f24200fbcb0 msgr2=0x7f24200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 -- 172.21.15.92:0/394277120 shutdown_connections 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 -- 172.21.15.92:0/394277120 wait complete. 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 Processor -- start 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 -- start start 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2420100740 0x7f2420197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 0x7f2420198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24201022f0 0x7f242019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f242010c3c0 con 0x7f2420101930 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f242010c240 con 0x7f24201022f0 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f24262c6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f242010c540 con 0x7f2420100740 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.159+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 0x7f2420198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2420100740 0x7f2420197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 0x7f2420198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50536/0 (socket says 172.21.15.92:50536) 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241f7fe640 1 -- 172.21.15.92:0/1628002730 learned_addr learned my addr 172.21.15.92:0/1628002730 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f2424863640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24201022f0 0x7f242019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241f7fe640 1 -- 172.21.15.92:0/1628002730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2420100740 msgr2=0x7f2420197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241f7fe640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2420100740 0x7f2420197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241f7fe640 1 -- 172.21.15.92:0/1628002730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24201022f0 msgr2=0x7f242019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241f7fe640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24201022f0 0x7f242019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241f7fe640 1 -- 172.21.15.92:0/1628002730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242019f710 con 0x7f2420101930 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241ffff640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2420100740 0x7f2420197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f2424863640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24201022f0 0x7f242019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241f7fe640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 0x7f2420198080 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2414009230 tx=0x7f241402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241d7fa640 1 -- 172.21.15.92:0/1628002730 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414041070 con 0x7f2420101930 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241d7fa640 1 -- 172.21.15.92:0/1628002730 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2414004450 con 0x7f2420101930 2024-03-19T01:54:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f24262c6640 1 -- 172.21.15.92:0/1628002730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242019f9a0 con 0x7f2420101930 2024-03-19T01:54:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f241d7fa640 1 -- 172.21.15.92:0/1628002730 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414003e20 con 0x7f2420101930 2024-03-19T01:54:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.160+0000 7f24262c6640 1 -- 172.21.15.92:0/1628002730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242019fe90 con 0x7f2420101930 2024-03-19T01:54:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.162+0000 7f241d7fa640 1 -- 172.21.15.92:0/1628002730 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f2414044070 con 0x7f2420101930 2024-03-19T01:54:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2428f93740 1 Processor -- start 2024-03-19T01:54:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2428f93740 1 -- start start 2024-03-19T01:54:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241d7fa640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f24080a28a0 0x7f24080a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2428f93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6c5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241d7fa640 1 -- 172.21.15.92:0/1628002730 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f24140f05d0 con 0x7f2420101930 2024-03-19T01:54:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2428f93740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2424863640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f24080a28a0 0x7f24080a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2428f93740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2428f93740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55957a629be0 con 0x55957a6c5af0 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2428f93740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55957a5d2960 con 0x55957a6cac60 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2428f93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55957a625c80 con 0x55957a6c5100 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:42418/0 (socket says 172.21.15.92:42418) 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241ffff640 1 -- 172.21.15.92:0/730209288 learned_addr learned my addr 172.21.15.92:0/730209288 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2424863640 1 --2- 172.21.15.92:0/730209288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241f7fe640 1 -- 172.21.15.92:0/730209288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 msgr2=0x55957a6c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241f7fe640 1 --2- 172.21.15.92:0/730209288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241f7fe640 1 -- 172.21.15.92:0/730209288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 msgr2=0x55957a6cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241f7fe640 1 --2- 172.21.15.92:0/730209288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241f7fe640 1 -- 172.21.15.92:0/730209288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957a5c9d20 con 0x55957a6c5af0 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241ffff640 1 --2- 172.21.15.92:0/730209288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f2424863640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f24080a28a0 0x7f24080a4d60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2418009fd0 tx=0x7f241800ff60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.163+0000 7f241f7fe640 1 --2- 172.21.15.92:0/730209288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6ca5f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24140ee0e0 tx=0x7f2414005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be9c77432a5aa0e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f24067fc640 1 -- 172.21.15.92:0/730209288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241404b040 con 0x55957a6c5af0 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f24067fc640 1 -- 172.21.15.92:0/730209288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24140c10d0 con 0x55957a6c5af0 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f24067fc640 1 -- 172.21.15.92:0/730209288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140c5290 con 0x55957a6c5af0 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f2428f93740 1 -- 172.21.15.92:0/730209288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 msgr2=0x55957a6ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f2428f93740 1 --2- 172.21.15.92:0/730209288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6ca5f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24140ee0e0 tx=0x7f2414005e50 comp rx=0 tx=0).stop 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f2428f93740 1 -- 172.21.15.92:0/730209288 shutdown_connections 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f2428f93740 1 --2- 172.21.15.92:0/730209288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f2428f93740 1 --2- 172.21.15.92:0/730209288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6ca5f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f2428f93740 1 --2- 172.21.15.92:0/730209288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f2428f93740 1 -- 172.21.15.92:0/730209288 >> 172.21.15.92:0/730209288 conn(0x55957a6c0670 msgr2=0x55957a6c0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.164+0000 7f2428f93740 1 -- 172.21.15.92:0/730209288 shutdown_connections 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 1 -- 172.21.15.92:0/730209288 wait complete. 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 1 -- start start 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 1 Processor -- start 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6c5af0 0x55957a6cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6cac60 0x55957a6d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55957a6cd9d0 con 0x55957a6c5100 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55957a5e5db0 con 0x55957a6c5af0 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55957a6cdb50 con 0x55957a6cac60 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2428f93740 10 client.0 _ll_register_callbacks cb 0x55957a5eae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f241ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f241ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50552/0 (socket says 172.21.15.92:50552) 2024-03-19T01:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f241ffff640 1 -- 172.21.15.92:0/1135155370 learned_addr learned my addr 172.21.15.92:0/1135155370 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f241f7fe640 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6c5af0 0x55957a6cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.165+0000 7f2424863640 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6cac60 0x55957a6d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f241ffff640 1 -- 172.21.15.92:0/1135155370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6cac60 msgr2=0x55957a6d63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f241ffff640 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6cac60 0x55957a6d63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f241ffff640 1 -- 172.21.15.92:0/1135155370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6c5af0 msgr2=0x55957a6cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f241ffff640 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6c5af0 0x55957a6cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f241ffff640 1 -- 172.21.15.92:0/1135155370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957a5e4c60 con 0x55957a6c5100 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f241ffff640 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6cee30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2410012b10 tx=0x7f2410012f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bb8c88ab222a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f23ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1135155370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241000fa20 con 0x55957a6c5100 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1135155370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f241000fbc0 con 0x55957a6c5100 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f2428f93740 1 -- 172.21.15.92:0/1135155370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24140c10d0 con 0x55957a6c5100 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1135155370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f241000fe00 con 0x55957a6c5100 2024-03-19T01:54:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f2428f93740 10 client.15459 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f2428f93740 1 -- 172.21.15.92:0/1135155370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55957a33e210 con 0x55957a6c5100 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1135155370 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24140c10d0 con 0x55957a6c5100 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.166+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1135155370 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f241001c540 con 0x55957a6c5100 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.167+0000 7f2428f93740 10 client.15459 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.167+0000 7f2428f93740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.167+0000 7f2428f93740 20 client.15459 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.167+0000 7f2428f93740 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55957a6dad80 0x55957a6dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.167+0000 7f2428f93740 4 client.15459 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.167+0000 7f2428f93740 1 -- 172.21.15.92:0/1135155370 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55957a637120 con 0x55957a6dad80 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.167+0000 7f241f7fe640 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55957a6dad80 0x55957a6dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.167+0000 7f241f7fe640 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55957a6dad80 0x55957a6dd1f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9965cd406f77bf09 server_cookie=5c1f93324974c931 in_seq=0 out_seq=0 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.167+0000 7f23ff7fe640 10 client.15459 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1135155370 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55957a637120 con 0x55957a6dad80 2024-03-19T01:54:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f23ff7fe640 10 client.15459 handle_command_reply: tid=0 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 client.15459 shutdown 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 -- 172.21.15.92:0/1135155370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 msgr2=0x55957a6cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6cee30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2410012b10 tx=0x7f2410012f10 comp rx=0 tx=0).stop 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 -- 172.21.15.92:0/1135155370 shutdown_connections 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55957a6dad80 0x55957a6dd1f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6cac60 0x55957a6d63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6c5af0 0x55957a6cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 --2- 172.21.15.92:0/1135155370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6cee30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 -- 172.21.15.92:0/1135155370 >> 172.21.15.92:0/1135155370 conn(0x55957a6c0670 msgr2=0x55957a6c18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f23ff7fe640 0 client.15459 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 -- 172.21.15.92:0/1135155370 shutdown_connections 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.168+0000 7f2428f93740 1 -- 172.21.15.92:0/1135155370 wait complete. 2024-03-19T01:54:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.169+0000 7f2428f93740 20 client.15459 trim_cache size 0 max 16384 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f2428f93740 1 Processor -- start 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f2428f93740 1 -- start start 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f2428f93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f2428f93740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f2428f93740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f2428f93740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a8bd0 con 0x55957a6c5af0 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f2428f93740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24201aa740 con 0x55957a6cac60 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f2428f93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a92a0 con 0x55957a6c5100 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6d6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50562/0 (socket says 172.21.15.92:50562) 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241f7fe640 1 -- 172.21.15.92:0/3391008780 learned_addr learned my addr 172.21.15.92:0/3391008780 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241ffff640 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f2424863640 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241f7fe640 1 -- 172.21.15.92:0/3391008780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 msgr2=0x55957a6bf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241f7fe640 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6bf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241f7fe640 1 -- 172.21.15.92:0/3391008780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 msgr2=0x55957a6ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241f7fe640 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241f7fe640 1 -- 172.21.15.92:0/3391008780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957a6bf900 con 0x55957a6c5af0 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.192+0000 7f241ffff640 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f241f7fe640 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6d6370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2414009430 tx=0x7f24140bcef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6982c0fa688dac62 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2405ffb640 1 -- 172.21.15.92:0/3391008780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241404b040 con 0x55957a6c5af0 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2405ffb640 1 -- 172.21.15.92:0/3391008780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24140c10d0 con 0x55957a6c5af0 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2405ffb640 1 -- 172.21.15.92:0/3391008780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140c5290 con 0x55957a6c5af0 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2428f93740 1 -- 172.21.15.92:0/3391008780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 msgr2=0x55957a6d6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2428f93740 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6d6370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2414009430 tx=0x7f24140bcef0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2428f93740 1 -- 172.21.15.92:0/3391008780 shutdown_connections 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2428f93740 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2428f93740 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5af0 0x55957a6d6370 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2428f93740 1 --2- 172.21.15.92:0/3391008780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5100 0x55957a6bf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2428f93740 1 -- 172.21.15.92:0/3391008780 >> 172.21.15.92:0/3391008780 conn(0x55957a6c0670 msgr2=0x55957a6c4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2428f93740 1 -- 172.21.15.92:0/3391008780 shutdown_connections 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.193+0000 7f2428f93740 1 -- 172.21.15.92:0/3391008780 wait complete. 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2428f93740 1 -- start start 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2428f93740 1 Processor -- start 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2428f93740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2428f93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5af0 0x55957a6d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2428f93740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6d5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2428f93740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f24201aa040 con 0x55957a6c5100 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2428f93740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a9970 con 0x55957a6cac60 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2428f93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24201a6bb0 con 0x55957a6c5af0 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2428f93740 10 client.0 _ll_register_callbacks cb 0x7f2410009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f241ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f241ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6d3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50570/0 (socket says 172.21.15.92:50570) 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f241ffff640 1 -- 172.21.15.92:0/1574629441 learned_addr learned my addr 172.21.15.92:0/1574629441 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f241f7fe640 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5af0 0x55957a6d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f2424863640 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6d5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f241ffff640 1 -- 172.21.15.92:0/1574629441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5af0 msgr2=0x55957a6d4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f241ffff640 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5af0 0x55957a6d4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f241ffff640 1 -- 172.21.15.92:0/1574629441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 msgr2=0x55957a6d5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f241ffff640 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6d5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.194+0000 7f241ffff640 1 -- 172.21.15.92:0/1574629441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957a40a620 con 0x55957a6c5100 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f241ffff640 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6d3d00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2410012ae0 tx=0x7f2410005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b125e9051b35ad58 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f23ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1574629441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241001c630 con 0x55957a6c5100 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1574629441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f24100050b0 con 0x55957a6c5100 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f2428f93740 1 -- 172.21.15.92:0/1574629441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24140c10d0 con 0x55957a6c5100 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1574629441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24100052b0 con 0x55957a6c5100 2024-03-19T01:54:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f2428f93740 10 client.15465 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f2428f93740 1 -- 172.21.15.92:0/1574629441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55957a741200 con 0x55957a6c5100 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1574629441 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24140c10d0 con 0x55957a6c5100 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1574629441 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f2410032480 con 0x55957a6c5100 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f2428f93740 10 client.15465 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f2428f93740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f2428f93740 20 client.15465 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f2428f93740 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55957a741bf0 0x55957a744010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f2428f93740 4 client.15465 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f2428f93740 1 -- 172.21.15.92:0/1574629441 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55957a631470 con 0x55957a741bf0 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.195+0000 7f241f7fe640 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55957a741bf0 0x55957a744010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.196+0000 7f23ff7fe640 10 client.15465 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.196+0000 7f241f7fe640 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55957a741bf0 0x55957a744010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8678514c73e16b server_cookie=3dce6d71ffadbba6 in_seq=0 out_seq=0 2024-03-19T01:54:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f23ff7fe640 1 -- 172.21.15.92:0/1574629441 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55957a631470 con 0x55957a741bf0 2024-03-19T01:54:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f23ff7fe640 10 client.15465 handle_command_reply: tid=0 2024-03-19T01:54:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 client.15465 shutdown 2024-03-19T01:54:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 -- 172.21.15.92:0/1574629441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 msgr2=0x55957a6d3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6d3d00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2410012ae0 tx=0x7f2410005cc0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 -- 172.21.15.92:0/1574629441 shutdown_connections 2024-03-19T01:54:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55957a741bf0 0x55957a744010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55957a6cac60 0x55957a6d5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f23ff7fe640 0 client.15465 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55957a6c5af0 0x55957a6d4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 --2- 172.21.15.92:0/1574629441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55957a6c5100 0x55957a6d3d00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 -- 172.21.15.92:0/1574629441 >> 172.21.15.92:0/1574629441 conn(0x55957a6c0670 msgr2=0x55957a6c6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 -- 172.21.15.92:0/1574629441 shutdown_connections 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 1 -- 172.21.15.92:0/1574629441 wait complete. 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.202+0000 7f2428f93740 20 client.15465 trim_cache size 0 max 16384 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:54:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:54:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:54:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 -- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f24080a28a0 msgr2=0x7f24080a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f24080a28a0 0x7f24080a4d60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2418009fd0 tx=0x7f241800ff60 comp rx=0 tx=0).stop 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 -- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 msgr2=0x7f2420198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 0x7f2420198080 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2414009230 tx=0x7f241402c650 comp rx=0 tx=0).stop 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 -- 172.21.15.92:0/1628002730 shutdown_connections 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24201022f0 0x7f242019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2420101930 0x7f2420198080 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f24080a28a0 0x7f24080a4d60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 --2- 172.21.15.92:0/1628002730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2420100740 0x7f2420197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 -- 172.21.15.92:0/1628002730 >> 172.21.15.92:0/1628002730 conn(0x7f24200fbcb0 msgr2=0x7f2420068ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 -- 172.21.15.92:0/1628002730 shutdown_connections 2024-03-19T01:54:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:40.203+0000 7f24262c6640 1 -- 172.21.15.92:0/1628002730 wait complete. 2024-03-19T01:54:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:40 smithi159 ceph-mon[67110]: pgmap v407: 97 pgs: 97 active+clean; 971 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:54:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:40 smithi092 ceph-mon[62574]: pgmap v407: 97 pgs: 97 active+clean; 971 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:54:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:40 smithi144 ceph-mon[67199]: pgmap v407: 97 pgs: 97 active+clean; 971 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:54:42.600 INFO:tasks.workunit.client.0.smithi092.stdout:1/600: open fd = 3 2024-03-19T01:54:42.600 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.600 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.600 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.600 INFO:tasks.workunit.client.0.smithi092.stdout:2/600: open fd = 3 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:3/600: open fd = 3 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:4/600: open fd = 3 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.601 INFO:tasks.workunit.client.0.smithi092.stdout:5/600: open fd = 3 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:6/600: open fd = 3 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:7/600: open fd = 3 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:8/600: open fd = 3 2024-03-19T01:54:42.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:9/600: open fd = 3 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:10/600: open fd = 3 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:11/600: open fd = 3 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:12/600: open fd = 3 2024-03-19T01:54:42.603 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:13/600: open fd = 3 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:14/600: open fd = 3 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:15/600: open fd = 3 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.604 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:16/600: open fd = 3 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:17/600: open fd = 3 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:18/600: open fd = 3 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:19/600: open fd = 3 2024-03-19T01:54:42.605 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:20/600: open fd = 3 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:21/600: open fd = 3 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:22/600: open fd = 3 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.606 INFO:tasks.workunit.client.0.smithi092.stdout:23/600: open fd = 3 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:24/600: open fd = 3 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:25/600: open fd = 3 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:26/600: open fd = 3 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.607 INFO:tasks.workunit.client.0.smithi092.stdout:27/600: open fd = 3 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:28/600: open fd = 3 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:29/600: open fd = 3 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:30/600: open fd = 3 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.608 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:31/600: open fd = 3 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:32/600: open fd = 3 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:33/600: open fd = 3 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:34/600: open fd = 3 2024-03-19T01:54:42.609 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:35/600: open fd = 3 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:36/600: open fd = 3 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:37/600: open fd = 3 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.610 INFO:tasks.workunit.client.0.smithi092.stdout:38/600: open fd = 3 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:39/600: open fd = 3 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:40/600: open fd = 3 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:41/600: open fd = 3 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.611 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:42/600: open fd = 3 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:43/600: open fd = 3 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:44/600: open fd = 3 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:45/600: open fd = 3 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.612 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:46/600: open fd = 3 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:47/600: open fd = 3 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:48/600: open fd = 3 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.613 INFO:tasks.workunit.client.0.smithi092.stdout:49/600: open fd = 3 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:50/600: open fd = 3 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:51/600: open fd = 3 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:52/600: open fd = 3 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.614 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:53/600: open fd = 3 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:54/600: open fd = 3 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:55/600: open fd = 3 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:56/600: open fd = 3 2024-03-19T01:54:42.615 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:57/600: open fd = 3 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:58/600: open fd = 3 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:59/600: open fd = 3 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:60/600: open fd = 3 2024-03-19T01:54:42.616 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:61/600: open fd = 3 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:62/600: open fd = 3 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:63/600: open fd = 3 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:54:42.617 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:54:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:42 smithi159 ceph-mon[67110]: pgmap v408: 97 pgs: 97 active+clean; 971 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-03-19T01:54:43.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:42 smithi092 ceph-mon[62574]: pgmap v408: 97 pgs: 97 active+clean; 971 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-03-19T01:54:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:42 smithi144 ceph-mon[67199]: pgmap v408: 97 pgs: 97 active+clean; 971 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-03-19T01:54:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:44 smithi159 ceph-mon[67110]: pgmap v409: 97 pgs: 97 active+clean; 1002 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T01:54:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:44 smithi092 ceph-mon[62574]: pgmap v409: 97 pgs: 97 active+clean; 1002 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T01:54:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:44 smithi144 ceph-mon[67199]: pgmap v409: 97 pgs: 97 active+clean; 1002 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T01:54:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:46 smithi159 ceph-mon[67110]: pgmap v410: 97 pgs: 97 active+clean; 1002 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:54:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:46 smithi092 ceph-mon[62574]: pgmap v410: 97 pgs: 97 active+clean; 1002 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:54:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:46 smithi144 ceph-mon[67199]: pgmap v410: 97 pgs: 97 active+clean; 1002 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:54:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:48 smithi159 ceph-mon[67110]: pgmap v411: 97 pgs: 97 active+clean; 1002 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:54:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:48 smithi092 ceph-mon[62574]: pgmap v411: 97 pgs: 97 active+clean; 1002 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:54:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:48 smithi144 ceph-mon[67199]: pgmap v411: 97 pgs: 97 active+clean; 1002 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:54:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:50 smithi159 ceph-mon[67110]: pgmap v412: 97 pgs: 97 active+clean; 1020 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:54:51.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:50 smithi092 ceph-mon[62574]: pgmap v412: 97 pgs: 97 active+clean; 1020 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:54:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:50 smithi144 ceph-mon[67199]: pgmap v412: 97 pgs: 97 active+clean; 1020 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:54:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:52 smithi159 ceph-mon[67110]: pgmap v413: 97 pgs: 97 active+clean; 1020 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:54:53.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:52 smithi092 ceph-mon[62574]: pgmap v413: 97 pgs: 97 active+clean; 1020 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:54:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:52 smithi144 ceph-mon[67199]: pgmap v413: 97 pgs: 97 active+clean; 1020 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:54:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.292+0000 7f72214ac640 1 -- 172.21.15.92:0/1240901558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c101930 msgr2=0x7f721c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.292+0000 7f72214ac640 1 --2- 172.21.15.92:0/1240901558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c101930 0x7f721c101db0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7204009900 tx=0x7f720402bc30 comp rx=0 tx=0).stop 2024-03-19T01:54:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.292+0000 7f72214ac640 1 -- 172.21.15.92:0/1240901558 shutdown_connections 2024-03-19T01:54:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.292+0000 7f72214ac640 1 --2- 172.21.15.92:0/1240901558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f721c1022f0 0x7f721c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.292+0000 7f72214ac640 1 --2- 172.21.15.92:0/1240901558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c101930 0x7f721c101db0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.292+0000 7f72214ac640 1 --2- 172.21.15.92:0/1240901558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721c100740 0x7f721c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.292+0000 7f72214ac640 1 -- 172.21.15.92:0/1240901558 >> 172.21.15.92:0/1240901558 conn(0x7f721c0fbcb0 msgr2=0x7f721c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.292+0000 7f72214ac640 1 -- 172.21.15.92:0/1240901558 shutdown_connections 2024-03-19T01:54:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.292+0000 7f72214ac640 1 -- 172.21.15.92:0/1240901558 wait complete. 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f72214ac640 1 Processor -- start 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f72214ac640 1 -- start start 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f72214ac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c100740 0x7f721c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f72214ac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f721c101930 0x7f721c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f72214ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721c1022f0 0x7f721c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f72214ac640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10c3a0 con 0x7f721c100740 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f72214ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10c220 con 0x7f721c1022f0 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f72214ac640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10c520 con 0x7f721c101930 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f721affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c100740 0x7f721c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f721affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c100740 0x7f721c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45250/0 (socket says 172.21.15.92:45250) 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f721affd640 1 -- 172.21.15.92:0/500900933 learned_addr learned my addr 172.21.15.92:0/500900933 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f7213fff640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f721c101930 0x7f721c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.293+0000 7f721b7fe640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721c1022f0 0x7f721c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f721affd640 1 -- 172.21.15.92:0/500900933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f721c101930 msgr2=0x7f721c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f721affd640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f721c101930 0x7f721c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f721affd640 1 -- 172.21.15.92:0/500900933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721c1022f0 msgr2=0x7f721c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f721affd640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721c1022f0 0x7f721c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f721affd640 1 -- 172.21.15.92:0/500900933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c19f6c0 con 0x7f721c100740 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f721affd640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c100740 0x7f721c197aa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f720c00e9c0 tx=0x7f720c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f7218ff9640 1 -- 172.21.15.92:0/500900933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c00cd80 con 0x7f721c100740 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f7218ff9640 1 -- 172.21.15.92:0/500900933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f720c0035f0 con 0x7f721c100740 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f72214ac640 1 -- 172.21.15.92:0/500900933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c19f9b0 con 0x7f721c100740 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f7218ff9640 1 -- 172.21.15.92:0/500900933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c010690 con 0x7f721c100740 2024-03-19T01:54:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.294+0000 7f72214ac640 1 -- 172.21.15.92:0/500900933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c19ff20 con 0x7f721c100740 2024-03-19T01:54:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.296+0000 7f7218ff9640 1 -- 172.21.15.92:0/500900933 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f720c0037a0 con 0x7f721c100740 2024-03-19T01:54:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.296+0000 7f7224172740 1 Processor -- start 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.296+0000 7f7224172740 1 -- start start 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f7224172740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fb100 0x562ca40fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f7224172740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fbaf0 0x562ca41005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f7224172740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 0x562ca41030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f7224172740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562ca405fbe0 con 0x562ca40fbaf0 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f7224172740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ca4008960 con 0x562ca40fb100 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f7224172740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ca405bc80 con 0x562ca4100c60 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f7218ff9640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72080a2920 0x7f72080a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fbaf0 0x562ca41005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fb100 0x562ca40fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fbaf0 0x562ca41005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45264/0 (socket says 172.21.15.92:45264) 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721affd640 1 -- 172.21.15.92:0/909058652 learned_addr learned my addr 172.21.15.92:0/909058652 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f7213fff640 1 --2- 172.21.15.92:0/909058652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 0x562ca41030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721b7fe640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72080a2920 0x7f72080a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f7218ff9640 1 -- 172.21.15.92:0/500900933 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f720c0ced80 con 0x7f721c100740 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721b7fe640 1 -- 172.21.15.92:0/909058652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 msgr2=0x562ca41030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721b7fe640 1 --2- 172.21.15.92:0/909058652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 0x562ca41030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721b7fe640 1 -- 172.21.15.92:0/909058652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fbaf0 msgr2=0x562ca41005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721b7fe640 1 --2- 172.21.15.92:0/909058652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fbaf0 0x562ca41005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.297+0000 7f721b7fe640 1 -- 172.21.15.92:0/909058652 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca3fffd20 con 0x562ca40fb100 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.298+0000 7f721b7fe640 1 --2- 172.21.15.92:0/909058652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fb100 0x562ca40fb520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7214005fd0 tx=0x7f721400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37499894a45b1615 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.298+0000 7f721b7fe640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72080a2920 0x7f72080a4de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f721400fc90 tx=0x7f721401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.298+0000 7f72117fa640 1 -- 172.21.15.92:0/909058652 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721401dbf0 con 0x562ca40fb100 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.298+0000 7f72117fa640 1 -- 172.21.15.92:0/909058652 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f721401dd90 con 0x562ca40fb100 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.298+0000 7f72117fa640 1 -- 172.21.15.92:0/909058652 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72140042b0 con 0x562ca40fb100 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 -- 172.21.15.92:0/909058652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fb100 msgr2=0x562ca40fb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 --2- 172.21.15.92:0/909058652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fb100 0x562ca40fb520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7214005fd0 tx=0x7f721400f790 comp rx=0 tx=0).stop 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 -- 172.21.15.92:0/909058652 shutdown_connections 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 --2- 172.21.15.92:0/909058652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 0x562ca41030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 --2- 172.21.15.92:0/909058652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fbaf0 0x562ca41005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 --2- 172.21.15.92:0/909058652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fb100 0x562ca40fb520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 -- 172.21.15.92:0/909058652 >> 172.21.15.92:0/909058652 conn(0x562ca40f6670 msgr2=0x562ca40f6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 -- 172.21.15.92:0/909058652 shutdown_connections 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 -- 172.21.15.92:0/909058652 wait complete. 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 -- start start 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.299+0000 7f7224172740 1 Processor -- start 2024-03-19T01:54:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7224172740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca4104f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7224172740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 0x562ca41054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7224172740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 0x562ca410c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7224172740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562ca4103bb0 con 0x562ca40fb100 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7224172740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ca401bdb0 con 0x562ca40fbaf0 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7224172740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ca4103d30 con 0x562ca4100c60 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7224172740 10 client.0 _ll_register_callbacks cb 0x562ca4020e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca4104f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca4104f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45276/0 (socket says 172.21.15.92:45276) 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721affd640 1 -- 172.21.15.92:0/129209964 learned_addr learned my addr 172.21.15.92:0/129209964 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7213fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 0x562ca41054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721b7fe640 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 0x562ca410c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721affd640 1 -- 172.21.15.92:0/129209964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 msgr2=0x562ca410c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721affd640 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 0x562ca410c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721affd640 1 -- 172.21.15.92:0/129209964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 msgr2=0x562ca41054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721affd640 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 0x562ca41054d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721affd640 1 -- 172.21.15.92:0/129209964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca401ac60 con 0x562ca40fb100 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7213fff640 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 0x562ca41054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f721affd640 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca4104f90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f720c0058c0 tx=0x7f720c0a1e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b2b6ba372bb367 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f72027fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f72027fc640 1 -- 172.21.15.92:0/129209964 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c09f880 con 0x562ca40fb100 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f72027fc640 1 -- 172.21.15.92:0/129209964 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f720c003b30 con 0x562ca40fb100 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f7224172740 1 -- 172.21.15.92:0/129209964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f721401dd90 con 0x562ca40fb100 2024-03-19T01:54:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.300+0000 7f72027fc640 1 -- 172.21.15.92:0/129209964 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f720c0a3ed0 con 0x562ca40fb100 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7224172740 10 client.15477 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7224172740 1 -- 172.21.15.92:0/129209964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ca3d74210 con 0x562ca40fb100 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f72027fc640 1 -- 172.21.15.92:0/129209964 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f721401dd90 con 0x562ca40fb100 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f72027fc640 1 -- 172.21.15.92:0/129209964 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f720c0975a0 con 0x562ca40fb100 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7224172740 10 client.15477 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7224172740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7224172740 20 client.15477 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7224172740 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562ca4110e60 0x562ca4113270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7224172740 4 client.15477 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:54:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7224172740 1 -- 172.21.15.92:0/129209964 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ca406d120 con 0x562ca4110e60 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7213fff640 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562ca4110e60 0x562ca4113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f72027fc640 10 client.15477 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.301+0000 7f7213fff640 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562ca4110e60 0x562ca4113270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3663ebb85a8247ec server_cookie=e16c8d21359c67e7 in_seq=0 out_seq=0 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.302+0000 7f72027fc640 1 -- 172.21.15.92:0/129209964 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ca406d120 con 0x562ca4110e60 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.302+0000 7f72027fc640 10 client.15477 handle_command_reply: tid=0 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.302+0000 7f7224172740 1 client.15477 shutdown 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 -- 172.21.15.92:0/129209964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 msgr2=0x562ca4104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca4104f90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f720c0058c0 tx=0x7f720c0a1e00 comp rx=0 tx=0).stop 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 -- 172.21.15.92:0/129209964 shutdown_connections 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562ca4110e60 0x562ca4113270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca4100c60 0x562ca410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 0x562ca41054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f72027fc640 0 client.15477 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 --2- 172.21.15.92:0/129209964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca4104f90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 -- 172.21.15.92:0/129209964 >> 172.21.15.92:0/129209964 conn(0x562ca40f6670 msgr2=0x562ca40f78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 -- 172.21.15.92:0/129209964 shutdown_connections 2024-03-19T01:54:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 1 -- 172.21.15.92:0/129209964 wait complete. 2024-03-19T01:54:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.303+0000 7f7224172740 20 client.15477 trim_cache size 0 max 16384 2024-03-19T01:54:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.317+0000 7f7224172740 1 Processor -- start 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.317+0000 7f7224172740 1 -- start start 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7224172740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40f6670 0x562ca41052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7224172740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fb100 0x562ca410c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7224172740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 0x562ca40fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7224172740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a8c30 con 0x562ca40f6670 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7224172740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1aa7a0 con 0x562ca40fbaf0 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7224172740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a9300 con 0x562ca40fb100 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40f6670 0x562ca41052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40f6670 0x562ca41052b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45290/0 (socket says 172.21.15.92:45290) 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721affd640 1 -- 172.21.15.92:0/1505390413 learned_addr learned my addr 172.21.15.92:0/1505390413 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7213fff640 1 --2- 172.21.15.92:0/1505390413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fb100 0x562ca410c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721b7fe640 1 --2- 172.21.15.92:0/1505390413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 0x562ca40fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721affd640 1 -- 172.21.15.92:0/1505390413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fb100 msgr2=0x562ca410c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721affd640 1 --2- 172.21.15.92:0/1505390413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fb100 0x562ca410c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721affd640 1 -- 172.21.15.92:0/1505390413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 msgr2=0x562ca40fc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721affd640 1 --2- 172.21.15.92:0/1505390413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 0x562ca40fc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721affd640 1 -- 172.21.15.92:0/1505390413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca3e40620 con 0x562ca40f6670 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f721affd640 1 --2- 172.21.15.92:0/1505390413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40f6670 0x562ca41052b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f720c0980b0 tx=0x7f720c099c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f892fb8a913866a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7210ff9640 1 -- 172.21.15.92:0/1505390413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c09be30 con 0x562ca40f6670 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7210ff9640 1 -- 172.21.15.92:0/1505390413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f720c09f880 con 0x562ca40f6670 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.318+0000 7f7210ff9640 1 -- 172.21.15.92:0/1505390413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c097480 con 0x562ca40f6670 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 -- 172.21.15.92:0/1505390413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40f6670 msgr2=0x562ca41052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 --2- 172.21.15.92:0/1505390413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40f6670 0x562ca41052b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f720c0980b0 tx=0x7f720c099c40 comp rx=0 tx=0).stop 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 -- 172.21.15.92:0/1505390413 shutdown_connections 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 --2- 172.21.15.92:0/1505390413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40fbaf0 0x562ca40fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 --2- 172.21.15.92:0/1505390413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fb100 0x562ca410c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 --2- 172.21.15.92:0/1505390413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40f6670 0x562ca41052b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 -- 172.21.15.92:0/1505390413 >> 172.21.15.92:0/1505390413 conn(0x562ca40f1d20 msgr2=0x562ca4104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 -- 172.21.15.92:0/1505390413 shutdown_connections 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 -- 172.21.15.92:0/1505390413 wait complete. 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 -- start start 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 Processor -- start 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40f6670 0x562ca4109de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.319+0000 7f7224172740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca410a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7224172740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fbaf0 0x562ca410b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7224172740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1aa0a0 con 0x562ca40fb100 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7224172740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a99d0 con 0x562ca40f6670 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7224172740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a6c10 con 0x562ca40fbaf0 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7224172740 10 client.0 _ll_register_callbacks cb 0x7f720c09b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7213fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca410a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7213fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca410a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45306/0 (socket says 172.21.15.92:45306) 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7213fff640 1 -- 172.21.15.92:0/1048242420 learned_addr learned my addr 172.21.15.92:0/1048242420 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:54:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f721b7fe640 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fbaf0 0x562ca410b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f721affd640 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40f6670 0x562ca4109de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7213fff640 1 -- 172.21.15.92:0/1048242420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fbaf0 msgr2=0x562ca410b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7213fff640 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fbaf0 0x562ca410b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7213fff640 1 -- 172.21.15.92:0/1048242420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40f6670 msgr2=0x562ca4109de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7213fff640 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40f6670 0x562ca4109de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7213fff640 1 -- 172.21.15.92:0/1048242420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca410b920 con 0x562ca40fb100 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7213fff640 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca410a320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f720402c7b0 tx=0x7f720402ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b74b83189ffe6ce server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f72027fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f72027fc640 1 -- 172.21.15.92:0/1048242420 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040040c0 con 0x562ca40fb100 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f72027fc640 1 -- 172.21.15.92:0/1048242420 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7204004260 con 0x562ca40fb100 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7224172740 1 -- 172.21.15.92:0/1048242420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f720c09f880 con 0x562ca40fb100 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f72027fc640 1 -- 172.21.15.92:0/1048242420 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7204004460 con 0x562ca40fb100 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.320+0000 7f7224172740 10 client.15483 fetch_fsmap learned FSMap version 28 2024-03-19T01:54:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f7224172740 1 -- 172.21.15.92:0/1048242420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ca4177460 con 0x562ca40fb100 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f72027fc640 1 -- 172.21.15.92:0/1048242420 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f720c09f880 con 0x562ca40fb100 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f72027fc640 1 -- 172.21.15.92:0/1048242420 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f7204046070 con 0x562ca40fb100 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f7224172740 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f7224172740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f7224172740 20 client.15483 populate_metadata read hostname 'smithi092' 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f7224172740 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562ca4100c60 0x562ca4179f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f7224172740 4 client.15483 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f7224172740 1 -- 172.21.15.92:0/1048242420 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ca4067470 con 0x562ca4100c60 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f721affd640 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562ca4100c60 0x562ca4179f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f72027fc640 10 client.15483 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.321+0000 7f721affd640 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562ca4100c60 0x562ca4179f00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3d3440c56fc407 server_cookie=44b3e7186b436201 in_seq=0 out_seq=0 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f72027fc640 1 -- 172.21.15.92:0/1048242420 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ca4067470 con 0x562ca4100c60 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f72027fc640 10 client.15483 handle_command_reply: tid=0 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 client.15483 shutdown 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 -- 172.21.15.92:0/1048242420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 msgr2=0x562ca410a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca410a320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f720402c7b0 tx=0x7f720402ce40 comp rx=0 tx=0).stop 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 -- 172.21.15.92:0/1048242420 shutdown_connections 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x562ca4100c60 0x562ca4179f00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ca40fbaf0 0x562ca410b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562ca40fb100 0x562ca410a320 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f72027fc640 0 client.15483 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:54:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 --2- 172.21.15.92:0/1048242420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ca40f6670 0x562ca4109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 -- 172.21.15.92:0/1048242420 >> 172.21.15.92:0/1048242420 conn(0x562ca40f1d20 msgr2=0x562ca41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 -- 172.21.15.92:0/1048242420 shutdown_connections 2024-03-19T01:54:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 1 -- 172.21.15.92:0/1048242420 wait complete. 2024-03-19T01:54:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.328+0000 7f7224172740 20 client.15483 trim_cache size 0 max 16384 2024-03-19T01:54:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:54:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:54:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:54:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:54:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 -- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72080a2920 msgr2=0x7f72080a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72080a2920 0x7f72080a4de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f721400fc90 tx=0x7f721401e000 comp rx=0 tx=0).stop 2024-03-19T01:54:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 -- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c100740 msgr2=0x7f721c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:54:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c100740 0x7f721c197aa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f720c00e9c0 tx=0x7f720c00ee80 comp rx=0 tx=0).stop 2024-03-19T01:54:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 -- 172.21.15.92:0/500900933 shutdown_connections 2024-03-19T01:54:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721c1022f0 0x7f721c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f72080a2920 0x7f72080a4de0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f721c101930 0x7f721c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 --2- 172.21.15.92:0/500900933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f721c100740 0x7f721c197aa0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:54:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 -- 172.21.15.92:0/500900933 >> 172.21.15.92:0/500900933 conn(0x7f721c0fbcb0 msgr2=0x7f721c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:54:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 -- 172.21.15.92:0/500900933 shutdown_connections 2024-03-19T01:54:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:54:53.329+0000 7f72214ac640 1 -- 172.21.15.92:0/500900933 wait complete. 2024-03-19T01:54:54.899 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:54 smithi144 ceph-mon[67199]: pgmap v414: 97 pgs: 97 active+clean; 1.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:54:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:54 smithi159 ceph-mon[67110]: pgmap v414: 97 pgs: 97 active+clean; 1.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:54:55.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:54 smithi092 ceph-mon[62574]: pgmap v414: 97 pgs: 97 active+clean; 1.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:54:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:56 smithi159 ceph-mon[67110]: pgmap v415: 97 pgs: 97 active+clean; 1.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-03-19T01:54:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:56 smithi092 ceph-mon[62574]: pgmap v415: 97 pgs: 97 active+clean; 1.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-03-19T01:54:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:56 smithi144 ceph-mon[67199]: pgmap v415: 97 pgs: 97 active+clean; 1.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-03-19T01:54:58.958 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:54:58 smithi159 ceph-mon[67110]: pgmap v416: 97 pgs: 97 active+clean; 1.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:54:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:54:58 smithi092 ceph-mon[62574]: pgmap v416: 97 pgs: 97 active+clean; 1.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:54:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:54:58 smithi144 ceph-mon[67199]: pgmap v416: 97 pgs: 97 active+clean; 1.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:55:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:00 smithi159 ceph-mon[67110]: pgmap v417: 97 pgs: 97 active+clean; 1.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:55:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:00 smithi092 ceph-mon[62574]: pgmap v417: 97 pgs: 97 active+clean; 1.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:55:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:00 smithi144 ceph-mon[67199]: pgmap v417: 97 pgs: 97 active+clean; 1.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:55:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:02 smithi159 ceph-mon[67110]: pgmap v418: 97 pgs: 97 active+clean; 1.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:02 smithi092 ceph-mon[62574]: pgmap v418: 97 pgs: 97 active+clean; 1.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:02 smithi144 ceph-mon[67199]: pgmap v418: 97 pgs: 97 active+clean; 1.0 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:04 smithi159 ceph-mon[67110]: pgmap v419: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:04 smithi092 ceph-mon[62574]: pgmap v419: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:05.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:04 smithi144 ceph-mon[67199]: pgmap v419: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.416+0000 7f14fe1b2640 1 -- 172.21.15.92:0/1894120838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f8100a00 msgr2=0x7f14f8100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.416+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/1894120838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f8100a00 0x7f14f8100e80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009900 tx=0x7f14ec02bc30 comp rx=0 tx=0).stop 2024-03-19T01:55:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.416+0000 7f14fe1b2640 1 -- 172.21.15.92:0/1894120838 shutdown_connections 2024-03-19T01:55:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.416+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/1894120838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f81013c0 0x7f14f8108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.416+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/1894120838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f8100a00 0x7f14f8100e80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.416+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/1894120838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14f8101cd0 0x7f14f81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.416+0000 7f14fe1b2640 1 -- 172.21.15.92:0/1894120838 >> 172.21.15.92:0/1894120838 conn(0x7f14f80fb470 msgr2=0x7f14f80fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.416+0000 7f14fe1b2640 1 -- 172.21.15.92:0/1894120838 shutdown_connections 2024-03-19T01:55:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.416+0000 7f14fe1b2640 1 -- 172.21.15.92:0/1894120838 wait complete. 2024-03-19T01:55:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14fe1b2640 1 Processor -- start 2024-03-19T01:55:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14fe1b2640 1 -- start start 2024-03-19T01:55:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14fe1b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f8100a00 0x7f14f8196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14fe1b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f81013c0 0x7f14f8196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14fe1b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14f8101cd0 0x7f14f819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14fe1b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f14f810c3d0 con 0x7f14f81013c0 2024-03-19T01:55:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14fe1b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14f810c250 con 0x7f14f8100a00 2024-03-19T01:55:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14fe1b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14f810c550 con 0x7f14f8101cd0 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f81013c0 0x7f14f8196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f81013c0 0x7f14f8196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48310/0 (socket says 172.21.15.92:48310) 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14f6ffd640 1 -- 172.21.15.92:0/3354596437 learned_addr learned my addr 172.21.15.92:0/3354596437 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14f77fe640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f8100a00 0x7f14f8196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.417+0000 7f14f7fff640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14f8101cd0 0x7f14f819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f6ffd640 1 -- 172.21.15.92:0/3354596437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14f8101cd0 msgr2=0x7f14f819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14f8101cd0 0x7f14f819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f6ffd640 1 -- 172.21.15.92:0/3354596437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f8100a00 msgr2=0x7f14f8196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f8100a00 0x7f14f8196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f6ffd640 1 -- 172.21.15.92:0/3354596437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f819e5b0 con 0x7f14f81013c0 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f77fe640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f8100a00 0x7f14f8196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f81013c0 0x7f14f8196ed0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f14ec02efd0 tx=0x7f14ec02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f4ff9640 1 -- 172.21.15.92:0/3354596437 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec004450 con 0x7f14f81013c0 2024-03-19T01:55:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14fe1b2640 1 -- 172.21.15.92:0/3354596437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f80ff8f0 con 0x7f14f81013c0 2024-03-19T01:55:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f4ff9640 1 -- 172.21.15.92:0/3354596437 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec0054e0 con 0x7f14f81013c0 2024-03-19T01:55:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14fe1b2640 1 -- 172.21.15.92:0/3354596437 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f80ffe30 con 0x7f14f81013c0 2024-03-19T01:55:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.418+0000 7f14f4ff9640 1 -- 172.21.15.92:0/3354596437 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec03b440 con 0x7f14f81013c0 2024-03-19T01:55:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.420+0000 7f14f4ff9640 1 -- 172.21.15.92:0/3354596437 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f14ec004f80 con 0x7f14f81013c0 2024-03-19T01:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f1500d9e740 1 Processor -- start 2024-03-19T01:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f4ff9640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14e00a28a0 0x7f14e00a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f1500d9e740 1 -- start start 2024-03-19T01:55:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a2240520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a22455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f7fff640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14e00a28a0 0x7f14e00a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 0x5570a22480f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f1500d9e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5570a21a4be0 con 0x5570a2240af0 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f1500d9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5570a214d960 con 0x5570a2240100 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f1500d9e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5570a21a0c80 con 0x5570a2245c60 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f4ff9640 1 -- 172.21.15.92:0/3354596437 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f14ec0f2570 con 0x7f14f81013c0 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a2240520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a2240520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:56608/0 (socket says 172.21.15.92:56608) 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a22455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f77fe640 1 -- 172.21.15.92:0/3939728757 learned_addr learned my addr 172.21.15.92:0/3939728757 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f7fff640 1 --2- 172.21.15.92:0/3939728757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 0x5570a22480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f77fe640 1 -- 172.21.15.92:0/3939728757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 msgr2=0x5570a22480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f77fe640 1 --2- 172.21.15.92:0/3939728757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 0x5570a22480f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f77fe640 1 -- 172.21.15.92:0/3939728757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 msgr2=0x5570a22455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f77fe640 1 --2- 172.21.15.92:0/3939728757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a22455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f77fe640 1 -- 172.21.15.92:0/3939728757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a2144d20 con 0x5570a2240100 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f7fff640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14e00a28a0 0x7f14e00a4d60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009fd0 tx=0x7f14f0013770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.421+0000 7f14f77fe640 1 --2- 172.21.15.92:0/3939728757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a2240520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14e80114f0 tx=0x7f14e80119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b819a066d47fdcce server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f14ddffb640 1 -- 172.21.15.92:0/3939728757 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8004020 con 0x5570a2240100 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f14ddffb640 1 -- 172.21.15.92:0/3939728757 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e80041c0 con 0x5570a2240100 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f14ddffb640 1 -- 172.21.15.92:0/3939728757 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800f930 con 0x5570a2240100 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f1500d9e740 1 -- 172.21.15.92:0/3939728757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 msgr2=0x5570a2240520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f1500d9e740 1 --2- 172.21.15.92:0/3939728757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a2240520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14e80114f0 tx=0x7f14e80119b0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f1500d9e740 1 -- 172.21.15.92:0/3939728757 shutdown_connections 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f1500d9e740 1 --2- 172.21.15.92:0/3939728757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 0x5570a22480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f1500d9e740 1 --2- 172.21.15.92:0/3939728757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a22455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f1500d9e740 1 --2- 172.21.15.92:0/3939728757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a2240520 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f1500d9e740 1 -- 172.21.15.92:0/3939728757 >> 172.21.15.92:0/3939728757 conn(0x5570a223b670 msgr2=0x5570a223ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f1500d9e740 1 -- 172.21.15.92:0/3939728757 shutdown_connections 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.422+0000 7f1500d9e740 1 -- 172.21.15.92:0/3939728757 wait complete. 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f1500d9e740 1 -- start start 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f1500d9e740 1 Processor -- start 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a224a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a224a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2245c60 0x5570a22515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f1500d9e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5570a2248c10 con 0x5570a2245c60 2024-03-19T01:55:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f1500d9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5570a2160db0 con 0x5570a2240100 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f1500d9e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5570a2248d90 con 0x5570a2240af0 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f1500d9e740 10 client.0 _ll_register_callbacks cb 0x5570a2165e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a224a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2245c60 0x5570a22515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a224a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:37824/0 (socket says 172.21.15.92:37824) 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f6ffd640 1 -- 172.21.15.92:0/1035869296 learned_addr learned my addr 172.21.15.92:0/1035869296 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f77fe640 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a224a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f6ffd640 1 -- 172.21.15.92:0/1035869296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 msgr2=0x5570a224a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a224a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f6ffd640 1 -- 172.21.15.92:0/1035869296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2245c60 msgr2=0x5570a22515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2245c60 0x5570a22515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f6ffd640 1 -- 172.21.15.92:0/1035869296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a215fc60 con 0x5570a2240af0 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.423+0000 7f14f7fff640 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2245c60 0x5570a22515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a224a580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0f03f0 tx=0x7f14ec0be900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d29da9710f4b3631 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f14d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f14d6ffd640 1 -- 172.21.15.92:0/1035869296 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0bef20 con 0x5570a2240af0 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f1500d9e740 1 -- 172.21.15.92:0/1035869296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e80041c0 con 0x5570a2240af0 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f14d6ffd640 1 -- 172.21.15.92:0/1035869296 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f14ec0bf180 con 0x5570a2240af0 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f14d6ffd640 1 -- 172.21.15.92:0/1035869296 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14ec0c3070 con 0x5570a2240af0 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f1500d9e740 10 client.25381 fetch_fsmap learned FSMap version 28 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f1500d9e740 1 -- 172.21.15.92:0/1035869296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570a1eb9210 con 0x5570a2240af0 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f14d6ffd640 1 -- 172.21.15.92:0/1035869296 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e80041c0 con 0x5570a2240af0 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f14d6ffd640 1 -- 172.21.15.92:0/1035869296 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f14ec0c2030 con 0x5570a2240af0 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f1500d9e740 10 client.25381 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f1500d9e740 10 client.25381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f1500d9e740 20 client.25381 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f1500d9e740 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5570a2255f40 0x5570a22583c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f1500d9e740 4 client.25381 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.424+0000 7f1500d9e740 1 -- 172.21.15.92:0/1035869296 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570a21b2120 con 0x5570a2255f40 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.425+0000 7f14f77fe640 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5570a2255f40 0x5570a22583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.425+0000 7f14d6ffd640 10 client.25381 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.425+0000 7f14f77fe640 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5570a2255f40 0x5570a22583c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4571e88c5cee2a4c server_cookie=343e0582d6c2670 in_seq=0 out_seq=0 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f14d6ffd640 1 -- 172.21.15.92:0/1035869296 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5570a21b2120 con 0x5570a2255f40 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f14d6ffd640 10 client.25381 handle_command_reply: tid=0 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 client.25381 shutdown 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 -- 172.21.15.92:0/1035869296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 msgr2=0x5570a224a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a224a580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0f03f0 tx=0x7f14ec0be900 comp rx=0 tx=0).stop 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 -- 172.21.15.92:0/1035869296 shutdown_connections 2024-03-19T01:55:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5570a2255f40 0x5570a22583c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2245c60 0x5570a22515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f14d6ffd640 0 client.25381 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a224a580 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 --2- 172.21.15.92:0/1035869296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a224a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 -- 172.21.15.92:0/1035869296 >> 172.21.15.92:0/1035869296 conn(0x5570a223b670 msgr2=0x5570a223c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 -- 172.21.15.92:0/1035869296 shutdown_connections 2024-03-19T01:55:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 1 -- 172.21.15.92:0/1035869296 wait complete. 2024-03-19T01:55:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.426+0000 7f1500d9e740 20 client.25381 trim_cache size 0 max 16384 2024-03-19T01:55:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.440+0000 7f1500d9e740 1 Processor -- start 2024-03-19T01:55:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.440+0000 7f1500d9e740 1 -- start start 2024-03-19T01:55:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240100 0x5570a2245060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a2251580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2245c60 0x5570a2241380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f1500d9e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a69d0 con 0x5570a2240100 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f1500d9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a8540 con 0x5570a2245c60 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f1500d9e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a70a0 con 0x5570a2240af0 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240100 0x5570a2245060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240100 0x5570a2245060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48334/0 (socket says 172.21.15.92:48334) 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f77fe640 1 -- 172.21.15.92:0/3387757974 learned_addr learned my addr 172.21.15.92:0/3387757974 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/3387757974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a2251580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f7fff640 1 --2- 172.21.15.92:0/3387757974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2245c60 0x5570a2241380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f77fe640 1 -- 172.21.15.92:0/3387757974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 msgr2=0x5570a2251580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f77fe640 1 --2- 172.21.15.92:0/3387757974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a2251580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f77fe640 1 -- 172.21.15.92:0/3387757974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2245c60 msgr2=0x5570a2241380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f77fe640 1 --2- 172.21.15.92:0/3387757974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2245c60 0x5570a2241380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f77fe640 1 -- 172.21.15.92:0/3387757974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a1f85620 con 0x5570a2240100 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14f77fe640 1 --2- 172.21.15.92:0/3387757974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240100 0x5570a2245060 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f14e8014040 tx=0x7f14e8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62da21f6329719b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14dd7fa640 1 -- 172.21.15.92:0/3387757974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8003dc0 con 0x5570a2240100 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14dd7fa640 1 -- 172.21.15.92:0/3387757974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8004120 con 0x5570a2240100 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.441+0000 7f14dd7fa640 1 -- 172.21.15.92:0/3387757974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e80052d0 con 0x5570a2240100 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 -- 172.21.15.92:0/3387757974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240100 msgr2=0x5570a2245060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 --2- 172.21.15.92:0/3387757974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240100 0x5570a2245060 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f14e8014040 tx=0x7f14e8011440 comp rx=0 tx=0).stop 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 -- 172.21.15.92:0/3387757974 shutdown_connections 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 --2- 172.21.15.92:0/3387757974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2245c60 0x5570a2241380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 --2- 172.21.15.92:0/3387757974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2240af0 0x5570a2251580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 --2- 172.21.15.92:0/3387757974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240100 0x5570a2245060 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 -- 172.21.15.92:0/3387757974 >> 172.21.15.92:0/3387757974 conn(0x5570a223b670 msgr2=0x5570a2252010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 -- 172.21.15.92:0/3387757974 shutdown_connections 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 -- 172.21.15.92:0/3387757974 wait complete. 2024-03-19T01:55:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 -- start start 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.442+0000 7f1500d9e740 1 Processor -- start 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a224ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a224f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f1500d9e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 0x5570a2250360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f1500d9e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a7e40 con 0x5570a2240af0 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f1500d9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a7770 con 0x5570a2240100 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f1500d9e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14f81a49b0 con 0x5570a2245c60 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f1500d9e740 10 client.0 _ll_register_callbacks cb 0x7f14ec009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a224f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a224f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48344/0 (socket says 172.21.15.92:48344) 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a224ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f6ffd640 1 -- 172.21.15.92:0/276989252 learned_addr learned my addr 172.21.15.92:0/276989252 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f7fff640 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 0x5570a2250360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f6ffd640 1 -- 172.21.15.92:0/276989252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 msgr2=0x5570a2250360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 0x5570a2250360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f6ffd640 1 -- 172.21.15.92:0/276989252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 msgr2=0x5570a224ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a224ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f6ffd640 1 -- 172.21.15.92:0/276989252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a2250950 con 0x5570a2240af0 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f77fe640 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a224ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14f6ffd640 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a224f350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0bb8f0 tx=0x7f14ec009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7416cf7f3c2efd server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14d6ffd640 1 -- 172.21.15.92:0/276989252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0bef20 con 0x5570a2240af0 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14d6ffd640 1 -- 172.21.15.92:0/276989252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f14ec0c7230 con 0x5570a2240af0 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f1500d9e740 1 -- 172.21.15.92:0/276989252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e8004120 con 0x5570a2240af0 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.443+0000 7f14d6ffd640 1 -- 172.21.15.92:0/276989252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14ec0c76c0 con 0x5570a2240af0 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f1500d9e740 10 client.15501 fetch_fsmap learned FSMap version 28 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f1500d9e740 1 -- 172.21.15.92:0/276989252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570a22bc470 con 0x5570a2240af0 2024-03-19T01:55:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f14d6ffd640 1 -- 172.21.15.92:0/276989252 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8004120 con 0x5570a2240af0 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f14d6ffd640 1 -- 172.21.15.92:0/276989252 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f14ec0c2040 con 0x5570a2240af0 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f1500d9e740 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f1500d9e740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f1500d9e740 20 client.15501 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f1500d9e740 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5570a22bca40 0x5570a22bee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f1500d9e740 4 client.15501 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f1500d9e740 1 -- 172.21.15.92:0/276989252 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570a21ac470 con 0x5570a22bca40 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f14f77fe640 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5570a22bca40 0x5570a22bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f14d6ffd640 10 client.15501 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.444+0000 7f14f77fe640 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5570a22bca40 0x5570a22bee60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ed5af5422bb8d6 server_cookie=30ffb8b90572c609 in_seq=0 out_seq=0 2024-03-19T01:55:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f14d6ffd640 1 -- 172.21.15.92:0/276989252 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570a21ac470 con 0x5570a22bca40 2024-03-19T01:55:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f14d6ffd640 10 client.15501 handle_command_reply: tid=0 2024-03-19T01:55:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 client.15501 shutdown 2024-03-19T01:55:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 -- 172.21.15.92:0/276989252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 msgr2=0x5570a224f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a224f350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0bb8f0 tx=0x7f14ec009560 comp rx=0 tx=0).stop 2024-03-19T01:55:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 -- 172.21.15.92:0/276989252 shutdown_connections 2024-03-19T01:55:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5570a22bca40 0x5570a22bee60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5570a2245c60 0x5570a2250360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5570a2240af0 0x5570a224f350 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f14d6ffd640 0 client.15501 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 --2- 172.21.15.92:0/276989252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570a2240100 0x5570a224ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 -- 172.21.15.92:0/276989252 >> 172.21.15.92:0/276989252 conn(0x5570a223b670 msgr2=0x5570a223e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 -- 172.21.15.92:0/276989252 shutdown_connections 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 1 -- 172.21.15.92:0/276989252 wait complete. 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.451+0000 7f1500d9e740 20 client.15501 trim_cache size 0 max 16384 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:55:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:55:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 -- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14e00a28a0 msgr2=0x7f14e00a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14e00a28a0 0x7f14e00a4d60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009fd0 tx=0x7f14f0013770 comp rx=0 tx=0).stop 2024-03-19T01:55:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 -- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f81013c0 msgr2=0x7f14f8196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f81013c0 0x7f14f8196ed0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f14ec02efd0 tx=0x7f14ec02c680 comp rx=0 tx=0).stop 2024-03-19T01:55:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 -- 172.21.15.92:0/3354596437 shutdown_connections 2024-03-19T01:55:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f14e00a28a0 0x7f14e00a4d60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14f8101cd0 0x7f14f819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14f81013c0 0x7f14f8196ed0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 --2- 172.21.15.92:0/3354596437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f8100a00 0x7f14f8196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 -- 172.21.15.92:0/3354596437 >> 172.21.15.92:0/3354596437 conn(0x7f14f80fb470 msgr2=0x7f14f8104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 -- 172.21.15.92:0/3354596437 shutdown_connections 2024-03-19T01:55:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:06.452+0000 7f14fe1b2640 1 -- 172.21.15.92:0/3354596437 wait complete. 2024-03-19T01:55:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:06 smithi159 ceph-mon[67110]: pgmap v420: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T01:55:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:06 smithi092 ceph-mon[62574]: pgmap v420: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T01:55:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:06 smithi144 ceph-mon[67199]: pgmap v420: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T01:55:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:08 smithi159 ceph-mon[67110]: pgmap v421: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-03-19T01:55:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:08 smithi092 ceph-mon[62574]: pgmap v421: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-03-19T01:55:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:08 smithi144 ceph-mon[67199]: pgmap v421: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-03-19T01:55:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:10 smithi159 ceph-mon[67110]: pgmap v422: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:55:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:10 smithi092 ceph-mon[62574]: pgmap v422: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:55:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:10 smithi144 ceph-mon[67199]: pgmap v422: 97 pgs: 97 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:55:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:12 smithi159 ceph-mon[67110]: pgmap v423: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:55:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:12 smithi092 ceph-mon[62574]: pgmap v423: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:55:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:12 smithi144 ceph-mon[67199]: pgmap v423: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:55:13.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:55:13.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:55:13.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:13 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:55:14.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:55:14.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:55:14.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:13 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:55:14.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:55:14.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:55:14.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:13 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:55:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:14 smithi159 ceph-mon[67110]: pgmap v424: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:55:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:14 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:55:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:14 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:55:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:14 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:55:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:14 smithi092 ceph-mon[62574]: pgmap v424: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:55:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:14 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:55:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:14 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:55:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:14 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:55:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:14 smithi144 ceph-mon[67199]: pgmap v424: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:55:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:14 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:55:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:14 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:55:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:14 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:55:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:16 smithi159 ceph-mon[67110]: pgmap v425: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:55:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:16 smithi092 ceph-mon[62574]: pgmap v425: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:55:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:16 smithi144 ceph-mon[67199]: pgmap v425: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:55:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:18 smithi159 ceph-mon[67110]: pgmap v426: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:55:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:18 smithi092 ceph-mon[62574]: pgmap v426: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:55:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:18 smithi144 ceph-mon[67199]: pgmap v426: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.550+0000 7f6d302de640 1 -- 172.21.15.92:0/1381916928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28100740 msgr2=0x7f6d28100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.550+0000 7f6d302de640 1 --2- 172.21.15.92:0/1381916928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28100740 0x7f6d28100b60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009900 tx=0x7f6d2002bc30 comp rx=0 tx=0).stop 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.550+0000 7f6d302de640 1 -- 172.21.15.92:0/1381916928 shutdown_connections 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.550+0000 7f6d302de640 1 --2- 172.21.15.92:0/1381916928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d281022f0 0x7f6d28108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.550+0000 7f6d302de640 1 --2- 172.21.15.92:0/1381916928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d28101930 0x7f6d28101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.550+0000 7f6d302de640 1 --2- 172.21.15.92:0/1381916928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28100740 0x7f6d28100b60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.550+0000 7f6d302de640 1 -- 172.21.15.92:0/1381916928 >> 172.21.15.92:0/1381916928 conn(0x7f6d280fbcb0 msgr2=0x7f6d280fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.550+0000 7f6d302de640 1 -- 172.21.15.92:0/1381916928 shutdown_connections 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.550+0000 7f6d302de640 1 -- 172.21.15.92:0/1381916928 wait complete. 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d302de640 1 Processor -- start 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d302de640 1 -- start start 2024-03-19T01:55:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d302de640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d28100740 0x7f6d28069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d302de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28101930 0x7f6d2819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d302de640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d281022f0 0x7f6d2819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d302de640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2810c3c0 con 0x7f6d28101930 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d302de640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2810c240 con 0x7f6d281022f0 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d302de640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2810c540 con 0x7f6d28100740 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2d879640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28101930 0x7f6d2819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2e07a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d28100740 0x7f6d28069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2d879640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28101930 0x7f6d2819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51284/0 (socket says 172.21.15.92:51284) 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2e87b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d281022f0 0x7f6d2819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2d879640 1 -- 172.21.15.92:0/1825756077 learned_addr learned my addr 172.21.15.92:0/1825756077 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2e07a640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d28100740 0x7f6d28069a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:54944/0 (socket says 172.21.15.92:54944) 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2d879640 1 -- 172.21.15.92:0/1825756077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d28100740 msgr2=0x7f6d28069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2d879640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d28100740 0x7f6d28069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2d879640 1 -- 172.21.15.92:0/1825756077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d281022f0 msgr2=0x7f6d2819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2d879640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d281022f0 0x7f6d2819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2d879640 1 -- 172.21.15.92:0/1825756077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2819f760 con 0x7f6d28101930 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d281022f0 0x7f6d2819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2e07a640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d28100740 0x7f6d28069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.551+0000 7f6d2d879640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28101930 0x7f6d2819c830 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00e9f0 tx=0x7f6d1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.552+0000 7f6d277fe640 1 -- 172.21.15.92:0/1825756077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00cd80 con 0x7f6d28101930 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.552+0000 7f6d302de640 1 -- 172.21.15.92:0/1825756077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2819f9f0 con 0x7f6d28101930 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.552+0000 7f6d277fe640 1 -- 172.21.15.92:0/1825756077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d1c010910 con 0x7f6d28101930 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.552+0000 7f6d277fe640 1 -- 172.21.15.92:0/1825756077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0188f0 con 0x7f6d28101930 2024-03-19T01:55:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.552+0000 7f6d302de640 1 -- 172.21.15.92:0/1825756077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2819ff40 con 0x7f6d28101930 2024-03-19T01:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.554+0000 7f6d277fe640 1 -- 172.21.15.92:0/1825756077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f6d1c010ac0 con 0x7f6d28101930 2024-03-19T01:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.554+0000 7f6d32f9b740 1 Processor -- start 2024-03-19T01:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.554+0000 7f6d32f9b740 1 -- start start 2024-03-19T01:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fc9c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fca40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d03fc00be0 con 0x55d03fc9c100 2024-03-19T01:55:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d03fba9960 con 0x55d03fc9caf0 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d03fbfcc80 con 0x55d03fca1c60 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fc9c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fc9c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51292/0 (socket says 172.21.15.92:51292) 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 -- 172.21.15.92:0/2541948800 learned_addr learned my addr 172.21.15.92:0/2541948800 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e07a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2d879640 1 --2- 172.21.15.92:0/2541948800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fca40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d277fe640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6d100a28a0 0x7f6d100a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 -- 172.21.15.92:0/2541948800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 msgr2=0x55d03fca40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/2541948800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fca40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 -- 172.21.15.92:0/2541948800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 msgr2=0x55d03fca15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/2541948800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 -- 172.21.15.92:0/2541948800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03fba0d20 con 0x55d03fc9c100 2024-03-19T01:55:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e07a640 1 --2- 172.21.15.92:0/2541948800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6d100a28a0 0x7f6d100a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d277fe640 1 -- 172.21.15.92:0/1825756077 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6d1c0cedc0 con 0x7f6d28101930 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.555+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/2541948800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fc9c520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011940 tx=0x7f6d18011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05d6ab6089f60d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d24bfe640 1 -- 172.21.15.92:0/2541948800 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18017070 con 0x55d03fc9c100 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6d100a28a0 0x7f6d100a4d60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d18023000 tx=0x7f6d18024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d24bfe640 1 -- 172.21.15.92:0/2541948800 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d18011250 con 0x55d03fc9c100 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d24bfe640 1 -- 172.21.15.92:0/2541948800 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18010c70 con 0x55d03fc9c100 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2541948800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 msgr2=0x55d03fc9c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2541948800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fc9c520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011940 tx=0x7f6d18011e00 comp rx=0 tx=0).stop 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2541948800 shutdown_connections 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2541948800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fca40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2541948800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2541948800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fc9c520 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.556+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2541948800 >> 172.21.15.92:0/2541948800 conn(0x55d03fc97670 msgr2=0x55d03fc97a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2541948800 shutdown_connections 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2541948800 wait complete. 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 -- start start 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 Processor -- start 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fca6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fcad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d03fca4c10 con 0x55d03fc9c100 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d03fbbcdb0 con 0x55d03fc9caf0 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d03fca4d90 con 0x55d03fca1c60 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d32f9b740 10 client.0 _ll_register_callbacks cb 0x55d03fbc1e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2e07a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fca6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2e07a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fca6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51306/0 (socket says 172.21.15.92:51306) 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2e07a640 1 -- 172.21.15.92:0/1536781251 learned_addr learned my addr 172.21.15.92:0/1536781251 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fcad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2d879640 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2e07a640 1 -- 172.21.15.92:0/1536781251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 msgr2=0x55d03fcad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2e07a640 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fcad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2e07a640 1 -- 172.21.15.92:0/1536781251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 msgr2=0x55d03fca6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2e07a640 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.557+0000 7f6d2e07a640 1 -- 172.21.15.92:0/1536781251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03fbbbc60 con 0x55d03fc9c100 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d2d879640 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d2e07a640 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fca6110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6d2002c7b0 tx=0x7f6d20005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b920a7d93ce3aa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d0dffb640 1 -- 172.21.15.92:0/1536781251 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20005d50 con 0x55d03fc9c100 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d0dffb640 1 -- 172.21.15.92:0/1536781251 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6d200048c0 con 0x55d03fc9c100 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d32f9b740 1 -- 172.21.15.92:0/1536781251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d18011250 con 0x55d03fc9c100 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d0dffb640 1 -- 172.21.15.92:0/1536781251 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d20005650 con 0x55d03fc9c100 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d32f9b740 10 client.15513 fetch_fsmap learned FSMap version 28 2024-03-19T01:55:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d32f9b740 1 -- 172.21.15.92:0/1536781251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d03f915210 con 0x55d03fc9c100 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d0dffb640 1 -- 172.21.15.92:0/1536781251 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d18011250 con 0x55d03fc9c100 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d0dffb640 1 -- 172.21.15.92:0/1536781251 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f6d20003b70 con 0x55d03fc9c100 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d32f9b740 10 client.15513 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d32f9b740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d32f9b740 20 client.15513 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d03fcb1f70 0x55d03fcb43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d32f9b740 4 client.15513 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.558+0000 7f6d32f9b740 1 -- 172.21.15.92:0/1536781251 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d03fc0e120 con 0x55d03fcb1f70 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.559+0000 7f6d2d879640 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d03fcb1f70 0x55d03fcb43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.559+0000 7f6d2d879640 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d03fcb1f70 0x55d03fcb43e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce68fd93642f534 server_cookie=8adca7f60dbdc917 in_seq=0 out_seq=0 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.559+0000 7f6d0dffb640 10 client.15513 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d0dffb640 1 -- 172.21.15.92:0/1536781251 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d03fc0e120 con 0x55d03fcb1f70 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d0dffb640 10 client.15513 handle_command_reply: tid=0 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 client.15513 shutdown 2024-03-19T01:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 -- 172.21.15.92:0/1536781251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 msgr2=0x55d03fca6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fca6110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6d2002c7b0 tx=0x7f6d20005250 comp rx=0 tx=0).stop 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 -- 172.21.15.92:0/1536781251 shutdown_connections 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d03fcb1f70 0x55d03fcb43e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fcad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fca6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/1536781251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9c100 0x55d03fca6110 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d0dffb640 0 client.15513 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 -- 172.21.15.92:0/1536781251 >> 172.21.15.92:0/1536781251 conn(0x55d03fc97670 msgr2=0x55d03fc988c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 -- 172.21.15.92:0/1536781251 shutdown_connections 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 1 -- 172.21.15.92:0/1536781251 wait complete. 2024-03-19T01:55:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.560+0000 7f6d32f9b740 20 client.15513 trim_cache size 0 max 16384 2024-03-19T01:55:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d32f9b740 1 Processor -- start 2024-03-19T01:55:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d32f9b740 1 -- start start 2024-03-19T01:55:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9c100 0x55d03fc9afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9caf0 0x55d03fcad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fca5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281a8cf0 con 0x55d03fc9caf0 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281aa860 con 0x55d03fc9c100 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281a93c0 con 0x55d03fca1c60 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2d879640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9caf0 0x55d03fcad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2e07a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9c100 0x55d03fc9afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2d879640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9caf0 0x55d03fcad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51322/0 (socket says 172.21.15.92:51322) 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2e07a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9c100 0x55d03fc9afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:48726/0 (socket says 172.21.15.92:48726) 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2d879640 1 -- 172.21.15.92:0/3279453671 learned_addr learned my addr 172.21.15.92:0/3279453671 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/3279453671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fca5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2d879640 1 -- 172.21.15.92:0/3279453671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 msgr2=0x55d03fca5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2d879640 1 --2- 172.21.15.92:0/3279453671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fca5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2d879640 1 -- 172.21.15.92:0/3279453671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9c100 msgr2=0x55d03fc9afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2d879640 1 --2- 172.21.15.92:0/3279453671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9c100 0x55d03fc9afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2d879640 1 -- 172.21.15.92:0/3279453671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03f9e1620 con 0x55d03fc9caf0 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2e07a640 1 --2- 172.21.15.92:0/3279453671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9c100 0x55d03fc9afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d2d879640 1 --2- 172.21.15.92:0/3279453671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9caf0 0x55d03fcad650 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c6a0 tx=0x7f6d1c0d12d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c5949984283e67 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d0ffff640 1 -- 172.21.15.92:0/3279453671 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c09f8c0 con 0x55d03fc9caf0 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.575+0000 7f6d0ffff640 1 -- 172.21.15.92:0/3279453671 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d1c0a3f10 con 0x55d03fc9caf0 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d0ffff640 1 -- 172.21.15.92:0/3279453671 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c09ac70 con 0x55d03fc9caf0 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 -- 172.21.15.92:0/3279453671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9caf0 msgr2=0x55d03fcad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/3279453671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9caf0 0x55d03fcad650 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c6a0 tx=0x7f6d1c0d12d0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 -- 172.21.15.92:0/3279453671 shutdown_connections 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/3279453671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fca1c60 0x55d03fca5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/3279453671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fc9caf0 0x55d03fcad650 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/3279453671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9c100 0x55d03fc9afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 -- 172.21.15.92:0/3279453671 >> 172.21.15.92:0/3279453671 conn(0x55d03fc97670 msgr2=0x55d03fc9a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 -- 172.21.15.92:0/3279453671 shutdown_connections 2024-03-19T01:55:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 -- 172.21.15.92:0/3279453671 wait complete. 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 -- start start 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.576+0000 7f6d32f9b740 1 Processor -- start 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fc9c100 0x55d03fcaaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fcaafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d32f9b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fca1c60 0x55d03fcabf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281aa160 con 0x55d03fca1c60 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281a9a90 con 0x55d03fc9caf0 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d32f9b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281a6cd0 con 0x55d03fc9c100 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d32f9b740 10 client.0 _ll_register_callbacks cb 0x7f6d20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e87b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fca1c60 0x55d03fcabf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e07a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fc9c100 0x55d03fcaaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e87b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fca1c60 0x55d03fcabf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51338/0 (socket says 172.21.15.92:51338) 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e07a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fc9c100 0x55d03fcaaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:54978/0 (socket says 172.21.15.92:54978) 2024-03-19T01:55:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e87b640 1 -- 172.21.15.92:0/2552131532 learned_addr learned my addr 172.21.15.92:0/2552131532 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2d879640 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fcaafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e87b640 1 -- 172.21.15.92:0/2552131532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fc9c100 msgr2=0x55d03fcaaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fc9c100 0x55d03fcaaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e87b640 1 -- 172.21.15.92:0/2552131532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 msgr2=0x55d03fcaafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fcaafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e87b640 1 -- 172.21.15.92:0/2552131532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03fcac550 con 0x55d03fca1c60 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2d879640 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fcaafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d2e87b640 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fca1c60 0x55d03fcabf60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55d03fc9d350 tx=0x7f6d18028650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96de5228e0512126 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d0dffb640 1 -- 172.21.15.92:0/2552131532 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18017040 con 0x55d03fca1c60 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2552131532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d1c0a3f10 con 0x55d03fca1c60 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.577+0000 7f6d0dffb640 1 -- 172.21.15.92:0/2552131532 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6d180289d0 con 0x55d03fca1c60 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d0dffb640 1 -- 172.21.15.92:0/2552131532 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d18011510 con 0x55d03fca1c60 2024-03-19T01:55:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d32f9b740 10 client.15519 fetch_fsmap learned FSMap version 28 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2552131532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d03fcad8d0 con 0x55d03fca1c60 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d0dffb640 1 -- 172.21.15.92:0/2552131532 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d1c0a3f10 con 0x55d03fca1c60 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d0dffb640 1 -- 172.21.15.92:0/2552131532 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f6d1801b070 con 0x55d03fca1c60 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d32f9b740 10 client.15519 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d32f9b740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d32f9b740 20 client.15519 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d03fd18970 0x55d03fd1ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d32f9b740 4 client.15519 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2552131532 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d03fc08470 con 0x55d03fd18970 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.578+0000 7f6d2e07a640 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d03fd18970 0x55d03fd1ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.579+0000 7f6d2e07a640 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d03fd18970 0x55d03fd1ad90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97093e1f63787a13 server_cookie=22d45754ba136b4 in_seq=0 out_seq=0 2024-03-19T01:55:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.579+0000 7f6d0dffb640 10 client.15519 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d0dffb640 1 -- 172.21.15.92:0/2552131532 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d03fc08470 con 0x55d03fd18970 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d0dffb640 10 client.15519 handle_command_reply: tid=0 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 client.15519 shutdown 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2552131532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fca1c60 msgr2=0x55d03fcabf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fca1c60 0x55d03fcabf60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55d03fc9d350 tx=0x7f6d18028650 comp rx=0 tx=0).stop 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2552131532 shutdown_connections 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d03fd18970 0x55d03fd1ad90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d03fca1c60 0x55d03fcabf60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03fc9caf0 0x55d03fcaafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d0dffb640 0 client.15519 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 --2- 172.21.15.92:0/2552131532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03fc9c100 0x55d03fcaaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2552131532 >> 172.21.15.92:0/2552131532 conn(0x55d03fc97670 msgr2=0x55d03fc9b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2552131532 shutdown_connections 2024-03-19T01:55:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 1 -- 172.21.15.92:0/2552131532 wait complete. 2024-03-19T01:55:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.585+0000 7f6d32f9b740 20 client.15519 trim_cache size 0 max 16384 2024-03-19T01:55:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:55:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:55:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:55:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:55:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 -- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6d100a28a0 msgr2=0x7f6d100a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6d100a28a0 0x7f6d100a4d60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d18023000 tx=0x7f6d18024000 comp rx=0 tx=0).stop 2024-03-19T01:55:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 -- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28101930 msgr2=0x7f6d2819c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28101930 0x7f6d2819c830 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00e9f0 tx=0x7f6d1c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 -- 172.21.15.92:0/1825756077 shutdown_connections 2024-03-19T01:55:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d281022f0 0x7f6d2819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d28101930 0x7f6d2819c830 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6d100a28a0 0x7f6d100a4d60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 --2- 172.21.15.92:0/1825756077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d28100740 0x7f6d28069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 -- 172.21.15.92:0/1825756077 >> 172.21.15.92:0/1825756077 conn(0x7f6d280fbcb0 msgr2=0x7f6d28104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 -- 172.21.15.92:0/1825756077 shutdown_connections 2024-03-19T01:55:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:19.586+0000 7f6d302de640 1 -- 172.21.15.92:0/1825756077 wait complete. 2024-03-19T01:55:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:20 smithi159 ceph-mon[67110]: pgmap v427: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T01:55:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:20 smithi092 ceph-mon[62574]: pgmap v427: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T01:55:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:20 smithi144 ceph-mon[67199]: pgmap v427: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T01:55:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:22 smithi159 ceph-mon[67110]: pgmap v428: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T01:55:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:22 smithi092 ceph-mon[62574]: pgmap v428: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T01:55:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:22 smithi144 ceph-mon[67199]: pgmap v428: 97 pgs: 97 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T01:55:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:24 smithi159 ceph-mon[67110]: pgmap v429: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:25.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:24 smithi092 ceph-mon[62574]: pgmap v429: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:25.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:24 smithi144 ceph-mon[67199]: pgmap v429: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:26 smithi159 ceph-mon[67110]: pgmap v430: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:26 smithi092 ceph-mon[62574]: pgmap v430: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:26 smithi144 ceph-mon[67199]: pgmap v430: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:28 smithi159 ceph-mon[67110]: pgmap v431: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:55:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:28 smithi092 ceph-mon[62574]: pgmap v431: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:55:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:28 smithi144 ceph-mon[67199]: pgmap v431: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:55:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:30 smithi159 ceph-mon[67110]: pgmap v432: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:55:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:30 smithi092 ceph-mon[62574]: pgmap v432: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:55:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:30 smithi144 ceph-mon[67199]: pgmap v432: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.673+0000 7f56a7f1b640 1 -- 172.21.15.92:0/2454717349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 msgr2=0x7f56a0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.673+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/2454717349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 0x7f56a0102ee0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5690009900 tx=0x7f569002bc30 comp rx=0 tx=0).stop 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.673+0000 7f56a7f1b640 1 -- 172.21.15.92:0/2454717349 shutdown_connections 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.673+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/2454717349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0103420 0x7f56a0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.673+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/2454717349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 0x7f56a0102ee0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.673+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/2454717349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a0101870 0x7f56a0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.673+0000 7f56a7f1b640 1 -- 172.21.15.92:0/2454717349 >> 172.21.15.92:0/2454717349 conn(0x7f56a00fcde0 msgr2=0x7f56a00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.673+0000 7f56a7f1b640 1 -- 172.21.15.92:0/2454717349 shutdown_connections 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.673+0000 7f56a7f1b640 1 -- 172.21.15.92:0/2454717349 wait complete. 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a7f1b640 1 Processor -- start 2024-03-19T01:55:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a7f1b640 1 -- start start 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a7f1b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a0101870 0x7f56a0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a7f1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 0x7f56a0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a7f1b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0103420 0x7f56a019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a7f1b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010d4f0 con 0x7f56a0102a60 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a7f1b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010d370 con 0x7f56a0101870 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a7f1b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010d670 con 0x7f56a0103420 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a54b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 0x7f56a0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a54b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 0x7f56a0197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44142/0 (socket says 172.21.15.92:44142) 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a54b6640 1 -- 172.21.15.92:0/3887822260 learned_addr learned my addr 172.21.15.92:0/3887822260 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a0101870 0x7f56a0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a64b8640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0103420 0x7f56a019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a54b6640 1 -- 172.21.15.92:0/3887822260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0103420 msgr2=0x7f56a019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a54b6640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0103420 0x7f56a019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.674+0000 7f56a54b6640 1 -- 172.21.15.92:0/3887822260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a0101870 msgr2=0x7f56a0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.675+0000 7f56a54b6640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a0101870 0x7f56a0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.675+0000 7f56a54b6640 1 -- 172.21.15.92:0/3887822260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a019f6c0 con 0x7f56a0102a60 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.675+0000 7f56a54b6640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 0x7f56a0197fe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5690009230 tx=0x7f569002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.675+0000 7f5696ffd640 1 -- 172.21.15.92:0/3887822260 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690041070 con 0x7f56a0102a60 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.675+0000 7f56a7f1b640 1 -- 172.21.15.92:0/3887822260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a019f950 con 0x7f56a0102a60 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.675+0000 7f5696ffd640 1 -- 172.21.15.92:0/3887822260 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5690004450 con 0x7f56a0102a60 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.675+0000 7f5696ffd640 1 -- 172.21.15.92:0/3887822260 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690003e20 con 0x7f56a0102a60 2024-03-19T01:55:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.675+0000 7f56a7f1b640 1 -- 172.21.15.92:0/3887822260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a019fe40 con 0x7f56a0102a60 2024-03-19T01:55:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.677+0000 7f5696ffd640 1 -- 172.21.15.92:0/3887822260 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f5690044070 con 0x7f56a0102a60 2024-03-19T01:55:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.677+0000 7f56aabe3740 1 Processor -- start 2024-03-19T01:55:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56aabe3740 1 -- start start 2024-03-19T01:55:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b292930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b297a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 0x55b87b29a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56aabe3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b87b1f2be0 con 0x55b87b292510 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56aabe3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b87b19b960 con 0x55b87b298070 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56aabe3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b87b1eec80 con 0x55b87b292f00 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a64b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b292930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a64b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b292930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44156/0 (socket says 172.21.15.92:44156) 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a64b8640 1 -- 172.21.15.92:0/1734657600 learned_addr learned my addr 172.21.15.92:0/1734657600 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a54b6640 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 0x55b87b29a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b297a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f5696ffd640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f568c0a28a0 0x7f568c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a5cb7640 1 -- 172.21.15.92:0/1734657600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 msgr2=0x55b87b29a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 0x55b87b29a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a5cb7640 1 -- 172.21.15.92:0/1734657600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 msgr2=0x55b87b292930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b292930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a5cb7640 1 -- 172.21.15.92:0/1734657600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87b192d20 con 0x55b87b292f00 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a54b6640 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 0x55b87b29a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a64b8640 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b292930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a64b8640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f568c0a28a0 0x7f568c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f5696ffd640 1 -- 172.21.15.92:0/3887822260 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f56900f05e0 con 0x7f56a0102a60 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.678+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b297a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5698009bf0 tx=0x7f5698011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caceadb4e2a535d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f568bfff640 1 -- 172.21.15.92:0/1734657600 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698009da0 con 0x55b87b292f00 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f56a64b8640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f568c0a28a0 0x7f568c0a4d60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55b87b293760 tx=0x7f569c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f568bfff640 1 -- 172.21.15.92:0/1734657600 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5698005020 con 0x55b87b292f00 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f568bfff640 1 -- 172.21.15.92:0/1734657600 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698004b60 con 0x55b87b292f00 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f56aabe3740 1 -- 172.21.15.92:0/1734657600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 msgr2=0x55b87b297a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b297a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5698009bf0 tx=0x7f5698011810 comp rx=0 tx=0).stop 2024-03-19T01:55:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f56aabe3740 1 -- 172.21.15.92:0/1734657600 shutdown_connections 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 0x55b87b29a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b297a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1734657600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b292930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f56aabe3740 1 -- 172.21.15.92:0/1734657600 >> 172.21.15.92:0/1734657600 conn(0x55b87b289670 msgr2=0x55b87b289a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.679+0000 7f56aabe3740 1 -- 172.21.15.92:0/1734657600 shutdown_connections 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 1 -- 172.21.15.92:0/1734657600 wait complete. 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 1 -- start start 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 1 Processor -- start 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292510 0x55b87b29c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292f00 0x55b87b29ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b87b29b020 con 0x55b87b298070 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b87b1aedb0 con 0x55b87b292f00 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b87b29b1a0 con 0x55b87b292510 2024-03-19T01:55:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56aabe3740 10 client.0 _ll_register_callbacks cb 0x55b87b1b3e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56a64b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56a54b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292f00 0x55b87b29ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56a64b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2a3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44158/0 (socket says 172.21.15.92:44158) 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56a5cb7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292510 0x55b87b29c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56a54b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292f00 0x55b87b29ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:37328/0 (socket says 172.21.15.92:37328) 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.680+0000 7f56a64b8640 1 -- 172.21.15.92:0/3745614103 learned_addr learned my addr 172.21.15.92:0/3745614103 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56a64b8640 1 -- 172.21.15.92:0/3745614103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292510 msgr2=0x55b87b29c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56a64b8640 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292510 0x55b87b29c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56a64b8640 1 -- 172.21.15.92:0/3745614103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292f00 msgr2=0x55b87b29ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56a64b8640 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292f00 0x55b87b29ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56a64b8640 1 -- 172.21.15.92:0/3745614103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87b1adc60 con 0x55b87b298070 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56a54b6640 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292f00 0x55b87b29ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56a64b8640 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2a3a40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f569c011a50 tx=0x7f569c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319d4babd53199e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56897fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:55:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56897fa640 1 -- 172.21.15.92:0/3745614103 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c01e0d0 con 0x55b87b298070 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56897fa640 1 -- 172.21.15.92:0/3745614103 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f569c01e940 con 0x55b87b298070 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56aabe3740 1 -- 172.21.15.92:0/3745614103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5698005020 con 0x55b87b298070 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56897fa640 1 -- 172.21.15.92:0/3745614103 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f569c01e270 con 0x55b87b298070 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56aabe3740 10 client.15531 fetch_fsmap learned FSMap version 28 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56aabe3740 1 -- 172.21.15.92:0/3745614103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b87af07210 con 0x55b87b298070 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.681+0000 7f56897fa640 1 -- 172.21.15.92:0/3745614103 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5698005020 con 0x55b87b298070 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56897fa640 1 -- 172.21.15.92:0/3745614103 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f569c01ccc0 con 0x55b87b298070 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56aabe3740 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56aabe3740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56aabe3740 20 client.15531 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56aabe3740 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b87b2a83c0 0x55b87b2aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56aabe3740 4 client.15531 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:55:32.690 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56aabe3740 1 -- 172.21.15.92:0/3745614103 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b87b200120 con 0x55b87b2a83c0 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b87b2a83c0 0x55b87b2aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56897fa640 10 client.15531 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.682+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b87b2a83c0 0x55b87b2aa800 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3baadabdaee7df server_cookie=8a9ccda6ce91ff5a in_seq=0 out_seq=0 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56897fa640 1 -- 172.21.15.92:0/3745614103 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b87b200120 con 0x55b87b2a83c0 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56897fa640 10 client.15531 handle_command_reply: tid=0 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 client.15531 shutdown 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 -- 172.21.15.92:0/3745614103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 msgr2=0x55b87b2a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2a3a40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f569c011a50 tx=0x7f569c004ab0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 -- 172.21.15.92:0/3745614103 shutdown_connections 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b87b2a83c0 0x55b87b2aa800 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2a3a40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292f00 0x55b87b29ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56897fa640 0 client.15531 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:32.691 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 --2- 172.21.15.92:0/3745614103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292510 0x55b87b29c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 -- 172.21.15.92:0/3745614103 >> 172.21.15.92:0/3745614103 conn(0x55b87b289670 msgr2=0x55b87b28a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 -- 172.21.15.92:0/3745614103 shutdown_connections 2024-03-19T01:55:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 1 -- 172.21.15.92:0/3745614103 wait complete. 2024-03-19T01:55:32.692 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.683+0000 7f56aabe3740 20 client.15531 trim_cache size 0 max 16384 2024-03-19T01:55:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56aabe3740 1 Processor -- start 2024-03-19T01:55:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56aabe3740 1 -- start start 2024-03-19T01:55:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292510 0x55b87b288ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b2a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56aabe3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a6a40 con 0x55b87b298070 2024-03-19T01:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56aabe3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a85b0 con 0x55b87b292510 2024-03-19T01:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56aabe3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a7110 con 0x55b87b292f00 2024-03-19T01:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a64b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a64b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44166/0 (socket says 172.21.15.92:44166) 2024-03-19T01:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a64b8640 1 -- 172.21.15.92:0/2010691756 learned_addr learned my addr 172.21.15.92:0/2010691756 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/2010691756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292510 0x55b87b288ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a54b6640 1 --2- 172.21.15.92:0/2010691756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b2a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a64b8640 1 -- 172.21.15.92:0/2010691756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 msgr2=0x55b87b2a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a64b8640 1 --2- 172.21.15.92:0/2010691756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b2a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a64b8640 1 -- 172.21.15.92:0/2010691756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292510 msgr2=0x55b87b288ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a64b8640 1 --2- 172.21.15.92:0/2010691756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292510 0x55b87b288ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.698+0000 7f56a64b8640 1 -- 172.21.15.92:0/2010691756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87afd3620 con 0x55b87b298070 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56a64b8640 1 --2- 172.21.15.92:0/2010691756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2940b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f569c0048a0 tx=0x7f569c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23f81db85d2a692 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f568b7fe640 1 -- 172.21.15.92:0/2010691756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c009500 con 0x55b87b298070 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f568b7fe640 1 -- 172.21.15.92:0/2010691756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569c02e030 con 0x55b87b298070 2024-03-19T01:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f568b7fe640 1 -- 172.21.15.92:0/2010691756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c01ea50 con 0x55b87b298070 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56aabe3740 1 -- 172.21.15.92:0/2010691756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 msgr2=0x55b87b2940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56aabe3740 1 --2- 172.21.15.92:0/2010691756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2940b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f569c0048a0 tx=0x7f569c005f70 comp rx=0 tx=0).stop 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56aabe3740 1 -- 172.21.15.92:0/2010691756 shutdown_connections 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56aabe3740 1 --2- 172.21.15.92:0/2010691756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b298070 0x55b87b2940b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56aabe3740 1 --2- 172.21.15.92:0/2010691756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b2a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56aabe3740 1 --2- 172.21.15.92:0/2010691756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b292510 0x55b87b288ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56aabe3740 1 -- 172.21.15.92:0/2010691756 >> 172.21.15.92:0/2010691756 conn(0x55b87b289670 msgr2=0x55b87b2939a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56aabe3740 1 -- 172.21.15.92:0/2010691756 shutdown_connections 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.699+0000 7f56aabe3740 1 -- 172.21.15.92:0/2010691756 wait complete. 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56aabe3740 1 -- start start 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56aabe3740 1 Processor -- start 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b2a1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b2a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56aabe3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 0x55b87b2a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56aabe3740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a7eb0 con 0x55b87b292510 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56aabe3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a77e0 con 0x55b87b298070 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56aabe3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a4a20 con 0x55b87b292f00 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56aabe3740 10 client.0 _ll_register_callbacks cb 0x7f569c003ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56a5cb7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b2a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56a5cb7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b2a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44170/0 (socket says 172.21.15.92:44170) 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56a5cb7640 1 -- 172.21.15.92:0/1007514295 learned_addr learned my addr 172.21.15.92:0/1007514295 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56a54b6640 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b2a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.700+0000 7f56a64b8640 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 0x55b87b2a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56a5cb7640 1 -- 172.21.15.92:0/1007514295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 msgr2=0x55b87b2a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b2a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56a5cb7640 1 -- 172.21.15.92:0/1007514295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 msgr2=0x55b87b2a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 0x55b87b2a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56a5cb7640 1 -- 172.21.15.92:0/1007514295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87b2a2d50 con 0x55b87b292510 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56a5cb7640 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b2a1260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5698009230 tx=0x7f5698009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4806944c2ff313a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56897fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56897fa640 1 -- 172.21.15.92:0/1007514295 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569802f020 con 0x55b87b292510 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56897fa640 1 -- 172.21.15.92:0/1007514295 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f5698005250 con 0x55b87b292510 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56aabe3740 1 -- 172.21.15.92:0/1007514295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569c02e030 con 0x55b87b292510 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56897fa640 1 -- 172.21.15.92:0/1007514295 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5698005470 con 0x55b87b292510 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56aabe3740 10 client.15537 fetch_fsmap learned FSMap version 28 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56aabe3740 1 -- 172.21.15.92:0/1007514295 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b87b30e8a0 con 0x55b87b292510 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56897fa640 1 -- 172.21.15.92:0/1007514295 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569c02e030 con 0x55b87b292510 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56897fa640 1 -- 172.21.15.92:0/1007514295 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f5698003610 con 0x55b87b292510 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56aabe3740 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.701+0000 7f56aabe3740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.702+0000 7f56aabe3740 20 client.15537 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.702+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b87b30f190 0x55b87b3115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.702+0000 7f56aabe3740 4 client.15537 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:55:32.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.702+0000 7f56aabe3740 1 -- 172.21.15.92:0/1007514295 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b87b1fa470 con 0x55b87b30f190 2024-03-19T01:55:32.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.702+0000 7f56a54b6640 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b87b30f190 0x55b87b3115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:32.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.702+0000 7f56a54b6640 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b87b30f190 0x55b87b3115b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4e6e401117a14e server_cookie=865b0eb16ed9692b in_seq=0 out_seq=0 2024-03-19T01:55:32.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.702+0000 7f56897fa640 10 client.15537 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56897fa640 1 -- 172.21.15.92:0/1007514295 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b87b1fa470 con 0x55b87b30f190 2024-03-19T01:55:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56897fa640 10 client.15537 handle_command_reply: tid=0 2024-03-19T01:55:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 client.15537 shutdown 2024-03-19T01:55:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 -- 172.21.15.92:0/1007514295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 msgr2=0x55b87b2a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b2a1260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5698009230 tx=0x7f5698009850 comp rx=0 tx=0).stop 2024-03-19T01:55:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 -- 172.21.15.92:0/1007514295 shutdown_connections 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b87b30f190 0x55b87b3115b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b87b298070 0x55b87b2a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b87b292f00 0x55b87b2a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56897fa640 0 client.15537 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 --2- 172.21.15.92:0/1007514295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b87b292510 0x55b87b2a1260 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 -- 172.21.15.92:0/1007514295 >> 172.21.15.92:0/1007514295 conn(0x55b87b289670 msgr2=0x55b87b294230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 -- 172.21.15.92:0/1007514295 shutdown_connections 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.708+0000 7f56aabe3740 1 -- 172.21.15.92:0/1007514295 wait complete. 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56aabe3740 20 client.15537 trim_cache size 0 max 16384 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:55:32.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 -- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f568c0a28a0 msgr2=0x7f568c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f568c0a28a0 0x7f568c0a4d60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55b87b293760 tx=0x7f569c004a40 comp rx=0 tx=0).stop 2024-03-19T01:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 -- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 msgr2=0x7f56a0197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 0x7f56a0197fe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5690009230 tx=0x7f569002c650 comp rx=0 tx=0).stop 2024-03-19T01:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 -- 172.21.15.92:0/3887822260 shutdown_connections 2024-03-19T01:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f568c0a28a0 0x7f568c0a4d60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0103420 0x7f56a019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56a0102a60 0x7f56a0197fe0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 --2- 172.21.15.92:0/3887822260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a0101870 0x7f56a0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 -- 172.21.15.92:0/3887822260 >> 172.21.15.92:0/3887822260 conn(0x7f56a00fcde0 msgr2=0x7f56a0105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 -- 172.21.15.92:0/3887822260 shutdown_connections 2024-03-19T01:55:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:32.709+0000 7f56a7f1b640 1 -- 172.21.15.92:0/3887822260 wait complete. 2024-03-19T01:55:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:32 smithi159 ceph-mon[67110]: pgmap v433: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:32 smithi092 ceph-mon[62574]: pgmap v433: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:32 smithi144 ceph-mon[67199]: pgmap v433: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:34 smithi159 ceph-mon[67110]: pgmap v434: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:34 smithi092 ceph-mon[62574]: pgmap v434: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:34 smithi144 ceph-mon[67199]: pgmap v434: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:36 smithi159 ceph-mon[67110]: pgmap v435: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T01:55:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:36 smithi092 ceph-mon[62574]: pgmap v435: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T01:55:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:36 smithi144 ceph-mon[67199]: pgmap v435: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T01:55:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:38 smithi159 ceph-mon[67110]: pgmap v436: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:55:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:38 smithi092 ceph-mon[62574]: pgmap v436: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:55:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:38 smithi144 ceph-mon[67199]: pgmap v436: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:55:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:40 smithi159 ceph-mon[67110]: pgmap v437: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:55:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:40 smithi092 ceph-mon[62574]: pgmap v437: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:55:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:40 smithi144 ceph-mon[67199]: pgmap v437: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:55:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:42 smithi159 ceph-mon[67110]: pgmap v438: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:55:43.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:42 smithi092 ceph-mon[62574]: pgmap v438: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:55:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:42 smithi144 ceph-mon[67199]: pgmap v438: 97 pgs: 97 active+clean; 1.2 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:55:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:44 smithi159 ceph-mon[67110]: pgmap v439: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:55:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:44 smithi092 ceph-mon[62574]: pgmap v439: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:55:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:44 smithi144 ceph-mon[67199]: pgmap v439: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.821+0000 7f94adee8640 1 -- 172.21.15.92:0/2934545574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8102a40 msgr2=0x7f94a8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.821+0000 7f94adee8640 1 --2- 172.21.15.92:0/2934545574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8102a40 0x7f94a8102ec0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f949c009900 tx=0x7f949c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.821+0000 7f94adee8640 1 -- 172.21.15.92:0/2934545574 shutdown_connections 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.821+0000 7f94adee8640 1 --2- 172.21.15.92:0/2934545574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a8103400 0x7f94a8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.821+0000 7f94adee8640 1 --2- 172.21.15.92:0/2934545574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8102a40 0x7f94a8102ec0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.821+0000 7f94adee8640 1 --2- 172.21.15.92:0/2934545574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94a8101850 0x7f94a8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.821+0000 7f94adee8640 1 -- 172.21.15.92:0/2934545574 >> 172.21.15.92:0/2934545574 conn(0x7f94a80fcde0 msgr2=0x7f94a80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.821+0000 7f94adee8640 1 -- 172.21.15.92:0/2934545574 shutdown_connections 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.821+0000 7f94adee8640 1 -- 172.21.15.92:0/2934545574 wait complete. 2024-03-19T01:55:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94adee8640 1 Processor -- start 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94adee8640 1 -- start start 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94adee8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94a8101850 0x7f94a8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94adee8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a8102a40 0x7f94a8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94adee8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8103400 0x7f94a819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94adee8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f94a810d4d0 con 0x7f94a8103400 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94adee8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94a810d350 con 0x7f94a8102a40 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94adee8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94a810d650 con 0x7f94a8101850 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8103400 0x7f94a819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94a8101850 0x7f94a8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8103400 0x7f94a819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44654/0 (socket says 172.21.15.92:44654) 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94a7fff640 1 -- 172.21.15.92:0/2968306671 learned_addr learned my addr 172.21.15.92:0/2968306671 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.822+0000 7f94a6ffd640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a8102a40 0x7f94a8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94a7fff640 1 -- 172.21.15.92:0/2968306671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94a8101850 msgr2=0x7f94a8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94a7fff640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94a8101850 0x7f94a8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94a7fff640 1 -- 172.21.15.92:0/2968306671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a8102a40 msgr2=0x7f94a8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94a7fff640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a8102a40 0x7f94a8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94a7fff640 1 -- 172.21.15.92:0/2968306671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a819f6d0 con 0x7f94a8103400 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94a7fff640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8103400 0x7f94a819f060 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f94a000eb10 tx=0x7f94a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94a4ff9640 1 -- 172.21.15.92:0/2968306671 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0004b20 con 0x7f94a8103400 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94a4ff9640 1 -- 172.21.15.92:0/2968306671 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a0004cc0 con 0x7f94a8103400 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94adee8640 1 -- 172.21.15.92:0/2968306671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a819f9c0 con 0x7f94a8103400 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94a4ff9640 1 -- 172.21.15.92:0/2968306671 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0010760 con 0x7f94a8103400 2024-03-19T01:55:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.823+0000 7f94adee8640 1 -- 172.21.15.92:0/2968306671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a819fe90 con 0x7f94a8103400 2024-03-19T01:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.825+0000 7f94a4ff9640 1 -- 172.21.15.92:0/2968306671 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f94a0005000 con 0x7f94a8103400 2024-03-19T01:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a4ff9640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f94900a28a0 0x7f94900a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94b0ba9740 1 Processor -- start 2024-03-19T01:55:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94b0ba9740 1 -- start start 2024-03-19T01:55:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a77fe640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f94900a28a0 0x7f94900a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a4ff9640 1 -- 172.21.15.92:0/2968306671 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f94a00cefc0 con 0x7f94a8103400 2024-03-19T01:55:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683a4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683a4f00 0x55cf683a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683aa070 0x55cf683ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cf68304be0 con 0x55cf683a4f00 2024-03-19T01:55:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf682ad960 con 0x55cf683a4510 2024-03-19T01:55:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf68300c80 con 0x55cf683aa070 2024-03-19T01:55:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683a4f00 0x55cf683a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683a4f00 0x55cf683a9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44658/0 (socket says 172.21.15.92:44658) 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a6ffd640 1 -- 172.21.15.92:0/2059524098 learned_addr learned my addr 172.21.15.92:0/2059524098 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a77fe640 1 --2- 172.21.15.92:0/2059524098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a7fff640 1 --2- 172.21.15.92:0/2059524098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683aa070 0x55cf683ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a6ffd640 1 -- 172.21.15.92:0/2059524098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683aa070 msgr2=0x55cf683ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a6ffd640 1 --2- 172.21.15.92:0/2059524098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683aa070 0x55cf683ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a6ffd640 1 -- 172.21.15.92:0/2059524098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 msgr2=0x55cf683a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a6ffd640 1 --2- 172.21.15.92:0/2059524098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a6ffd640 1 -- 172.21.15.92:0/2059524098 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf682a4d20 con 0x55cf683a4f00 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a77fe640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f94900a28a0 0x7f94900a4d60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94a81028a0 tx=0x7f9498011920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.826+0000 7f94a6ffd640 1 --2- 172.21.15.92:0/2059524098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683a4f00 0x55cf683a9a00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f949c009900 tx=0x7f949c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9740f733f83085 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f948dffb640 1 -- 172.21.15.92:0/2059524098 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c005d50 con 0x55cf683a4f00 2024-03-19T01:55:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f948dffb640 1 -- 172.21.15.92:0/2059524098 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949c02cb80 con 0x55cf683a4f00 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f948dffb640 1 -- 172.21.15.92:0/2059524098 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c004920 con 0x55cf683a4f00 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f94b0ba9740 1 -- 172.21.15.92:0/2059524098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683a4f00 msgr2=0x55cf683a9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/2059524098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683a4f00 0x55cf683a9a00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f949c009900 tx=0x7f949c009560 comp rx=0 tx=0).stop 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f94b0ba9740 1 -- 172.21.15.92:0/2059524098 shutdown_connections 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/2059524098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683aa070 0x55cf683ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/2059524098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683a4f00 0x55cf683a9a00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/2059524098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f94b0ba9740 1 -- 172.21.15.92:0/2059524098 >> 172.21.15.92:0/2059524098 conn(0x55cf6839b670 msgr2=0x55cf6839ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f94b0ba9740 1 -- 172.21.15.92:0/2059524098 shutdown_connections 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.827+0000 7f94b0ba9740 1 -- 172.21.15.92:0/2059524098 wait complete. 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94b0ba9740 1 -- start start 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94b0ba9740 1 Processor -- start 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf683aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683aa070 0x55cf683b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cf683acfc0 con 0x55cf683aa070 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cf682c0db0 con 0x55cf683a4510 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf683ad140 con 0x55cf683a4f00 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94b0ba9740 10 client.0 _ll_register_callbacks cb 0x55cf682c5e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683aa070 0x55cf683b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf683aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683aa070 0x55cf683b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44670/0 (socket says 172.21.15.92:44670) 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf683aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:37896/0 (socket says 172.21.15.92:37896) 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a7fff640 1 -- 172.21.15.92:0/223538574 learned_addr learned my addr 172.21.15.92:0/223538574 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a77fe640 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a7fff640 1 -- 172.21.15.92:0/223538574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 msgr2=0x55cf683aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a7fff640 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf683aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a7fff640 1 -- 172.21.15.92:0/223538574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 msgr2=0x55cf683ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a7fff640 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a7fff640 1 -- 172.21.15.92:0/223538574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf682bfc60 con 0x55cf683aa070 2024-03-19T01:55:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a6ffd640 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf683aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.828+0000 7f94a77fe640 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94a7fff640 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683aa070 0x55cf683b5a40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f94a009b670 tx=0x7f94a009b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdcb967e0b2385 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f9486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f9486ffd640 1 -- 172.21.15.92:0/223538574 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a00b3070 con 0x55cf683aa070 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f9486ffd640 1 -- 172.21.15.92:0/223538574 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f94a00a3c80 con 0x55cf683aa070 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94b0ba9740 1 -- 172.21.15.92:0/223538574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949c02cb80 con 0x55cf683aa070 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f9486ffd640 1 -- 172.21.15.92:0/223538574 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f94a009c070 con 0x55cf683aa070 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94b0ba9740 10 client.15549 fetch_fsmap learned FSMap version 29 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94b0ba9740 1 -- 172.21.15.92:0/223538574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cf68019210 con 0x55cf683aa070 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f9486ffd640 1 -- 172.21.15.92:0/223538574 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949c02cb80 con 0x55cf683aa070 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f9486ffd640 1 -- 172.21.15.92:0/223538574 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f94a00995b0 con 0x55cf683aa070 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94b0ba9740 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94b0ba9740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94b0ba9740 20 client.15549 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55cf683ba3f0 0x55cf683bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94b0ba9740 4 client.15549 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.829+0000 7f94b0ba9740 1 -- 172.21.15.92:0/223538574 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf68312120 con 0x55cf683ba3f0 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.830+0000 7f94a77fe640 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55cf683ba3f0 0x55cf683bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.830+0000 7f94a77fe640 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55cf683ba3f0 0x55cf683bc800 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7293d7cec9a8b1 server_cookie=19f92a12028dbb7f in_seq=0 out_seq=0 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.830+0000 7f9486ffd640 10 client.15549 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f9486ffd640 1 -- 172.21.15.92:0/223538574 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cf68312120 con 0x55cf683ba3f0 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f9486ffd640 10 client.15549 handle_command_reply: tid=0 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 client.15549 shutdown 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 -- 172.21.15.92:0/223538574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683aa070 msgr2=0x55cf683b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683aa070 0x55cf683b5a40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f94a009b670 tx=0x7f94a009b2d0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 -- 172.21.15.92:0/223538574 shutdown_connections 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55cf683ba3f0 0x55cf683bc800 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf683aa070 0x55cf683b5a40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf683aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/223538574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 -- 172.21.15.92:0/223538574 >> 172.21.15.92:0/223538574 conn(0x55cf6839b670 msgr2=0x55cf6839c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f9486ffd640 0 client.15549 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 -- 172.21.15.92:0/223538574 shutdown_connections 2024-03-19T01:55:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 1 -- 172.21.15.92:0/223538574 wait complete. 2024-03-19T01:55:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.831+0000 7f94b0ba9740 20 client.15549 trim_cache size 0 max 16384 2024-03-19T01:55:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94b0ba9740 1 Processor -- start 2024-03-19T01:55:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94b0ba9740 1 -- start start 2024-03-19T01:55:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf6839a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81a6940 con 0x55cf6839b670 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81a8480 con 0x55cf683a4510 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81a7010 con 0x55cf683a4f00 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683a94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44674/0 (socket says 172.21.15.92:44674) 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94a77fe640 1 -- 172.21.15.92:0/1286475145 learned_addr learned my addr 172.21.15.92:0/1286475145 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94a7fff640 1 --2- 172.21.15.92:0/1286475145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf6839a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.846+0000 7f94a6ffd640 1 --2- 172.21.15.92:0/1286475145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.847+0000 7f94a77fe640 1 -- 172.21.15.92:0/1286475145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 msgr2=0x55cf6839a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.847+0000 7f94a77fe640 1 --2- 172.21.15.92:0/1286475145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf6839a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.847+0000 7f94a77fe640 1 -- 172.21.15.92:0/1286475145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 msgr2=0x55cf683b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.847+0000 7f94a77fe640 1 --2- 172.21.15.92:0/1286475145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.847+0000 7f94a77fe640 1 -- 172.21.15.92:0/1286475145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf680e5620 con 0x55cf6839b670 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.847+0000 7f94a77fe640 1 --2- 172.21.15.92:0/1286475145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683a94f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55cf683a5760 tx=0x7f949801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcfd536043207a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.847+0000 7f948d7fa640 1 -- 172.21.15.92:0/1286475145 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949801a040 con 0x55cf6839b670 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.847+0000 7f948d7fa640 1 -- 172.21.15.92:0/1286475145 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949800fca0 con 0x55cf6839b670 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.847+0000 7f948d7fa640 1 -- 172.21.15.92:0/1286475145 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498020dc0 con 0x55cf6839b670 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 -- 172.21.15.92:0/1286475145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 msgr2=0x55cf683a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/1286475145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683a94f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55cf683a5760 tx=0x7f949801f000 comp rx=0 tx=0).stop 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 -- 172.21.15.92:0/1286475145 shutdown_connections 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/1286475145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4f00 0x55cf6839a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/1286475145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4510 0x55cf683b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/1286475145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683a94f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 -- 172.21.15.92:0/1286475145 >> 172.21.15.92:0/1286475145 conn(0x55cf683ae080 msgr2=0x55cf6839f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 -- 172.21.15.92:0/1286475145 shutdown_connections 2024-03-19T01:55:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 -- 172.21.15.92:0/1286475145 wait complete. 2024-03-19T01:55:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 -- start start 2024-03-19T01:55:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.848+0000 7f94b0ba9740 1 Processor -- start 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4510 0x55cf683b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94b0ba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4f00 0x55cf683b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81a7db0 con 0x55cf6839b670 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81a76e0 con 0x55cf683a4f00 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94b0ba9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94a81a4920 con 0x55cf683a4510 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94b0ba9740 10 client.0 _ll_register_callbacks cb 0x7f94a0097c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44690/0 (socket says 172.21.15.92:44690) 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a77fe640 1 -- 172.21.15.92:0/4156530520 learned_addr learned my addr 172.21.15.92:0/4156530520 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a7fff640 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4f00 0x55cf683b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a6ffd640 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4510 0x55cf683b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a77fe640 1 -- 172.21.15.92:0/4156530520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4510 msgr2=0x55cf683b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a77fe640 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4510 0x55cf683b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a77fe640 1 -- 172.21.15.92:0/4156530520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4f00 msgr2=0x55cf683b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a77fe640 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4f00 0x55cf683b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a77fe640 1 -- 172.21.15.92:0/4156530520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf683b50b0 con 0x55cf6839b670 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94a77fe640 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683b35c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9498009460 tx=0x7f9498009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafa58e687df694b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f9486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f9486ffd640 1 -- 172.21.15.92:0/4156530520 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949803a060 con 0x55cf6839b670 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f9486ffd640 1 -- 172.21.15.92:0/4156530520 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9498009040 con 0x55cf6839b670 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.849+0000 7f94b0ba9740 1 -- 172.21.15.92:0/4156530520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949800fca0 con 0x55cf6839b670 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f9486ffd640 1 -- 172.21.15.92:0/4156530520 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f949801a050 con 0x55cf6839b670 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f94b0ba9740 10 client.15555 fetch_fsmap learned FSMap version 29 2024-03-19T01:55:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f94b0ba9740 1 -- 172.21.15.92:0/4156530520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cf68420b60 con 0x55cf6839b670 2024-03-19T01:55:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f9486ffd640 1 -- 172.21.15.92:0/4156530520 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949800fca0 con 0x55cf6839b670 2024-03-19T01:55:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f9486ffd640 1 -- 172.21.15.92:0/4156530520 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f949801d030 con 0x55cf6839b670 2024-03-19T01:55:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f94b0ba9740 10 client.15555 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:55:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f94b0ba9740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f94b0ba9740 20 client.15555 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55cf683aa070 0x55cf68423540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f94b0ba9740 4 client.15555 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:55:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f94b0ba9740 1 -- 172.21.15.92:0/4156530520 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf6830c470 con 0x55cf683aa070 2024-03-19T01:55:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.850+0000 7f94a6ffd640 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55cf683aa070 0x55cf68423540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.851+0000 7f9486ffd640 10 client.15555 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.851+0000 7f94a6ffd640 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55cf683aa070 0x55cf68423540 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a1ca15b72e649f server_cookie=d2fb8bd6b548da0e in_seq=0 out_seq=0 2024-03-19T01:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f9486ffd640 1 -- 172.21.15.92:0/4156530520 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf6830c470 con 0x55cf683aa070 2024-03-19T01:55:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f9486ffd640 10 client.15555 handle_command_reply: tid=0 2024-03-19T01:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 client.15555 shutdown 2024-03-19T01:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 -- 172.21.15.92:0/4156530520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 msgr2=0x55cf683b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683b35c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9498009460 tx=0x7f9498009bc0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 -- 172.21.15.92:0/4156530520 shutdown_connections 2024-03-19T01:55:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55cf683aa070 0x55cf68423540 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cf683a4f00 0x55cf683b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf683a4510 0x55cf683b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 --2- 172.21.15.92:0/4156530520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf6839b670 0x55cf683b35c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 -- 172.21.15.92:0/4156530520 >> 172.21.15.92:0/4156530520 conn(0x55cf683ae080 msgr2=0x55cf6839f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f9486ffd640 0 client.15555 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 -- 172.21.15.92:0/4156530520 shutdown_connections 2024-03-19T01:55:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 1 -- 172.21.15.92:0/4156530520 wait complete. 2024-03-19T01:55:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.857+0000 7f94b0ba9740 20 client.15555 trim_cache size 0 max 16384 2024-03-19T01:55:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:55:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:55:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:55:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:55:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 -- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f94900a28a0 msgr2=0x7f94900a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f94900a28a0 0x7f94900a4d60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94a81028a0 tx=0x7f9498011920 comp rx=0 tx=0).stop 2024-03-19T01:55:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 -- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8103400 msgr2=0x7f94a819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8103400 0x7f94a819f060 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f94a000eb10 tx=0x7f94a000c2d0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 -- 172.21.15.92:0/2968306671 shutdown_connections 2024-03-19T01:55:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a8103400 0x7f94a819f060 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a8102a40 0x7f94a8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f94900a28a0 0x7f94900a4d60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 --2- 172.21.15.92:0/2968306671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94a8101850 0x7f94a8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 -- 172.21.15.92:0/2968306671 >> 172.21.15.92:0/2968306671 conn(0x7f94a80fcde0 msgr2=0x7f94a8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 -- 172.21.15.92:0/2968306671 shutdown_connections 2024-03-19T01:55:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:45.858+0000 7f94adee8640 1 -- 172.21.15.92:0/2968306671 wait complete. 2024-03-19T01:55:45.923 INFO:tasks.workunit.client.0.smithi092.stdout:64/600: open fd = 3 2024-03-19T01:55:45.923 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.923 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.923 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.923 INFO:tasks.workunit.client.0.smithi092.stdout:65/600: open fd = 3 2024-03-19T01:55:45.923 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.923 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:66/600: open fd = 3 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:67/600: open fd = 3 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:68/600: open fd = 3 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:69/600: open fd = 3 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:70/600: open fd = 3 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:71/600: open fd = 3 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:72/600: open fd = 3 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:73/600: open fd = 3 2024-03-19T01:55:45.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:74/600: open fd = 3 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:75/600: open fd = 3 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:76/600: open fd = 3 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.926 INFO:tasks.workunit.client.0.smithi092.stdout:77/600: open fd = 3 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:78/600: open fd = 3 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:79/600: open fd = 3 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:80/600: open fd = 3 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.927 INFO:tasks.workunit.client.0.smithi092.stdout:81/600: open fd = 3 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:82/600: open fd = 3 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:83/600: open fd = 3 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:84/600: open fd = 3 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.928 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:85/600: open fd = 3 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:86/600: open fd = 3 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:87/600: open fd = 3 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:88/600: open fd = 3 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:89/600: open fd = 3 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:90/600: open fd = 3 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:91/600: open fd = 3 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:92/600: open fd = 3 2024-03-19T01:55:45.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:93/600: open fd = 3 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:94/600: open fd = 3 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:95/600: open fd = 3 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:96/600: open fd = 3 2024-03-19T01:55:45.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:97/600: open fd = 3 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:98/600: open fd = 3 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:99/600: open fd = 3 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.932 INFO:tasks.workunit.client.0.smithi092.stdout:100/600: open fd = 3 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:101/600: open fd = 3 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:102/600: open fd = 3 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:103/600: open fd = 3 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.933 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:104/600: open fd = 3 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:105/600: open fd = 3 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:106/600: open fd = 3 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:107/600: open fd = 3 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:108/600: open fd = 3 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:109/600: open fd = 3 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:110/600: open fd = 3 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:111/600: open fd = 3 2024-03-19T01:55:45.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:112/600: open fd = 3 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:113/600: open fd = 3 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:114/600: open fd = 3 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:115/600: open fd = 3 2024-03-19T01:55:45.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:116/600: open fd = 3 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:117/600: open fd = 3 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:118/600: open fd = 3 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:119/600: open fd = 3 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:120/600: open fd = 3 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:121/600: open fd = 3 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:122/600: open fd = 3 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:123/600: open fd = 3 2024-03-19T01:55:45.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:124/600: open fd = 3 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:125/600: open fd = 3 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:126/600: open fd = 3 2024-03-19T01:55:45.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:55:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:46 smithi159 ceph-mon[67110]: pgmap v440: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:55:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:46 smithi159 ceph-mon[67110]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:55:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:46 smithi092 ceph-mon[62574]: pgmap v440: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:55:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:46 smithi092 ceph-mon[62574]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:55:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:46 smithi144 ceph-mon[67199]: pgmap v440: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:55:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:46 smithi144 ceph-mon[67199]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T01:55:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:48 smithi159 ceph-mon[67110]: pgmap v441: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-03-19T01:55:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:48 smithi092 ceph-mon[62574]: pgmap v441: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-03-19T01:55:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:48 smithi144 ceph-mon[67199]: pgmap v441: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-03-19T01:55:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:50 smithi159 ceph-mon[67110]: pgmap v442: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:55:51.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:50 smithi092 ceph-mon[62574]: pgmap v442: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:55:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:50 smithi144 ceph-mon[67199]: pgmap v442: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:55:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:52 smithi159 ceph-mon[67110]: pgmap v443: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:53.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:52 smithi092 ceph-mon[62574]: pgmap v443: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:52 smithi144 ceph-mon[67199]: pgmap v443: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:54 smithi159 ceph-mon[67110]: pgmap v444: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:55.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:54 smithi092 ceph-mon[62574]: pgmap v444: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:55.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:54 smithi144 ceph-mon[67199]: pgmap v444: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:55:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:56 smithi159 ceph-mon[67110]: pgmap v445: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:56 smithi092 ceph-mon[62574]: pgmap v445: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:56 smithi144 ceph-mon[67199]: pgmap v445: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:55:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.945+0000 7fd0c3e63640 1 -- 172.21.15.92:0/2395329823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 msgr2=0x7fd0bc108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.945+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/2395329823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 0x7fd0bc108b80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009ca0 tx=0x7fd0b402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.945+0000 7fd0c3e63640 1 -- 172.21.15.92:0/2395329823 shutdown_connections 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.945+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/2395329823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 0x7fd0bc108b80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.945+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/2395329823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0bc101930 0x7fd0bc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.945+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/2395329823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0bc100740 0x7fd0bc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.945+0000 7fd0c3e63640 1 -- 172.21.15.92:0/2395329823 >> 172.21.15.92:0/2395329823 conn(0x7fd0bc0fbcb0 msgr2=0x7fd0bc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.945+0000 7fd0c3e63640 1 -- 172.21.15.92:0/2395329823 shutdown_connections 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.945+0000 7fd0c3e63640 1 -- 172.21.15.92:0/2395329823 wait complete. 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c3e63640 1 Processor -- start 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c3e63640 1 -- start start 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c3e63640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0bc100740 0x7fd0bc197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c3e63640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0bc101930 0x7fd0bc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c3e63640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 0x7fd0bc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c3e63640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc10d4e0 con 0x7fd0bc1022f0 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c3e63640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc10d360 con 0x7fd0bc101930 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c3e63640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc10d660 con 0x7fd0bc100740 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c2400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 0x7fd0bc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c13fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0bc101930 0x7fd0bc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c2400640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 0x7fd0bc19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45552/0 (socket says 172.21.15.92:45552) 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c1bff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0bc100740 0x7fd0bc197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c2400640 1 -- 172.21.15.92:0/247462473 learned_addr learned my addr 172.21.15.92:0/247462473 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c2400640 1 -- 172.21.15.92:0/247462473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0bc100740 msgr2=0x7fd0bc197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c2400640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0bc100740 0x7fd0bc197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c2400640 1 -- 172.21.15.92:0/247462473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0bc101930 msgr2=0x7fd0bc198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c2400640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0bc101930 0x7fd0bc198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c2400640 1 -- 172.21.15.92:0/247462473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc19f6b0 con 0x7fd0bc1022f0 2024-03-19T01:55:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0c2400640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 0x7fd0bc19f040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4037040 tx=0x7fd0b4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.946+0000 7fd0baffd640 1 -- 172.21.15.92:0/247462473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4041020 con 0x7fd0bc1022f0 2024-03-19T01:55:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.947+0000 7fd0c3e63640 1 -- 172.21.15.92:0/247462473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0bc19f940 con 0x7fd0bc1022f0 2024-03-19T01:55:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.947+0000 7fd0baffd640 1 -- 172.21.15.92:0/247462473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b4034cd0 con 0x7fd0bc1022f0 2024-03-19T01:55:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.947+0000 7fd0baffd640 1 -- 172.21.15.92:0/247462473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4039040 con 0x7fd0bc1022f0 2024-03-19T01:55:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.947+0000 7fd0c3e63640 1 -- 172.21.15.92:0/247462473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0bc19fe80 con 0x7fd0bc1022f0 2024-03-19T01:55:58.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:55:58 smithi159 ceph-mon[67110]: pgmap v446: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:55:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.949+0000 7fd0baffd640 1 -- 172.21.15.92:0/247462473 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fd0b4044020 con 0x7fd0bc1022f0 2024-03-19T01:55:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c6a48740 1 Processor -- start 2024-03-19T01:55:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c6a48740 1 -- start start 2024-03-19T01:55:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed402c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed4031a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed4034500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560ed3f8cbe0 con 0x560ed4032070 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ed3f35960 con 0x560ed402cf00 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560ed3f88c80 con 0x560ed402c510 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0baffd640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0a40a2920 0x7fd0a40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c13fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed402c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c13fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed402c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:51110/0 (socket says 172.21.15.92:51110) 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c13fe640 1 -- 172.21.15.92:0/2135585535 learned_addr learned my addr 172.21.15.92:0/2135585535 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed4031a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c2400640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0a40a2920 0x7fd0a40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed4034500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0baffd640 1 -- 172.21.15.92:0/247462473 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd0b40f0870 con 0x7fd0bc1022f0 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c1bff640 1 -- 172.21.15.92:0/2135585535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 msgr2=0x560ed402c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed402c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c1bff640 1 -- 172.21.15.92:0/2135585535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 msgr2=0x560ed4034500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed4034500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c1bff640 1 -- 172.21.15.92:0/2135585535 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed3f2cd20 con 0x560ed402cf00 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.950+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed402c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.951+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed4031a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009bf0 tx=0x7fd0ac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571d70583f01230a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.951+0000 7fd0c2400640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0a40a2920 0x7fd0a40a4de0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd0b404c000 tx=0x7fd0b4034a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.951+0000 7fd0a3fff640 1 -- 172.21.15.92:0/2135585535 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac009da0 con 0x560ed402cf00 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.951+0000 7fd0a3fff640 1 -- 172.21.15.92:0/2135585535 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac005020 con 0x560ed402cf00 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.951+0000 7fd0a3fff640 1 -- 172.21.15.92:0/2135585535 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac004a50 con 0x560ed402cf00 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2135585535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 msgr2=0x560ed4031a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed4031a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009bf0 tx=0x7fd0ac011810 comp rx=0 tx=0).stop 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2135585535 shutdown_connections 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed4034500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed4031a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2135585535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed402c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2135585535 >> 172.21.15.92:0/2135585535 conn(0x560ed4023670 msgr2=0x560ed4023a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2135585535 shutdown_connections 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2135585535 wait complete. 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 -- start start 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 Processor -- start 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed40364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402cf00 0x560ed4036a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed4032070 0x560ed403da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560ed4035020 con 0x560ed402cf00 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ed3f48db0 con 0x560ed4032070 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.952+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560ed40351a0 con 0x560ed402c510 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c6a48740 10 client.0 _ll_register_callbacks cb 0x560ed3f4de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c1bff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402cf00 0x560ed4036a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c1bff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402cf00 0x560ed4036a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45564/0 (socket says 172.21.15.92:45564) 2024-03-19T01:55:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c1bff640 1 -- 172.21.15.92:0/2108638768 learned_addr learned my addr 172.21.15.92:0/2108638768 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed40364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed4032070 0x560ed403da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c1bff640 1 -- 172.21.15.92:0/2108638768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 msgr2=0x560ed40364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed40364f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c1bff640 1 -- 172.21.15.92:0/2108638768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed4032070 msgr2=0x560ed403da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed4032070 0x560ed403da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c1bff640 1 -- 172.21.15.92:0/2108638768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed3f47c60 con 0x560ed402cf00 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402cf00 0x560ed4036a30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0096e0 tx=0x7fd0ac004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdef28de49187c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2108638768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac029020 con 0x560ed402cf00 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2108638768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd0ac025470 con 0x560ed402cf00 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2108638768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ac005020 con 0x560ed402cf00 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2108638768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0ac009da0 con 0x560ed402cf00 2024-03-19T01:55:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c6a48740 10 client.15567 fetch_fsmap learned FSMap version 29 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.953+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2108638768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ed3ca1210 con 0x560ed402cf00 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.954+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2108638768 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac005020 con 0x560ed402cf00 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.954+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2108638768 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0ac019450 con 0x560ed402cf00 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.954+0000 7fd0c6a48740 10 client.15567 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.954+0000 7fd0c6a48740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.954+0000 7fd0c6a48740 20 client.15567 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.954+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560ed40421f0 0x560ed40445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.954+0000 7fd0c6a48740 4 client.15567 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.954+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2108638768 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ed3f9a120 con 0x560ed40421f0 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.954+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560ed40421f0 0x560ed40445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.955+0000 7fd0a17fa640 10 client.15567 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.955+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560ed40421f0 0x560ed40445e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1532ad8b1010cf server_cookie=b2e0c6e63ac436ce in_seq=0 out_seq=0 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.955+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2108638768 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560ed3f9a120 con 0x560ed40421f0 2024-03-19T01:55:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.955+0000 7fd0a17fa640 10 client.15567 handle_command_reply: tid=0 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.955+0000 7fd0c6a48740 1 client.15567 shutdown 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2108638768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402cf00 msgr2=0x560ed4036a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402cf00 0x560ed4036a30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0096e0 tx=0x7fd0ac004ac0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2108638768 shutdown_connections 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560ed40421f0 0x560ed40445e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed4032070 0x560ed403da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402cf00 0x560ed4036a30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2108638768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402c510 0x560ed40364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2108638768 >> 172.21.15.92:0/2108638768 conn(0x560ed4023670 msgr2=0x560ed40248c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0a17fa640 0 client.15567 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2108638768 shutdown_connections 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2108638768 wait complete. 2024-03-19T01:55:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.956+0000 7fd0c6a48740 20 client.15567 trim_cache size 0 max 16384 2024-03-19T01:55:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c6a48740 1 Processor -- start 2024-03-19T01:55:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c6a48740 1 -- start start 2024-03-19T01:55:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402c510 0x560ed4022930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402cf00 0x560ed403da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed402d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1a69a0 con 0x560ed4032070 2024-03-19T01:55:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1a84e0 con 0x560ed402c510 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1a7070 con 0x560ed402cf00 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c13fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed402d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c1bff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402cf00 0x560ed403da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.970+0000 7fd0c13fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed402d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45566/0 (socket says 172.21.15.92:45566) 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0c13fe640 1 -- 172.21.15.92:0/3051063072 learned_addr learned my addr 172.21.15.92:0/3051063072 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/3051063072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402c510 0x560ed4022930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0c1bff640 1 -- 172.21.15.92:0/3051063072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402c510 msgr2=0x560ed4022930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/3051063072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402c510 0x560ed4022930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0c1bff640 1 -- 172.21.15.92:0/3051063072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 msgr2=0x560ed402d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/3051063072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed402d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0c1bff640 1 -- 172.21.15.92:0/3051063072 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed3d6d620 con 0x560ed402cf00 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/3051063072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed402d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/3051063072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402cf00 0x560ed403da30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009250 tx=0x7fd0ac009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41313f8d8bc310df server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0a37fe640 1 -- 172.21.15.92:0/3051063072 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac029070 con 0x560ed402cf00 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0a37fe640 1 -- 172.21.15.92:0/3051063072 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac016070 con 0x560ed402cf00 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.971+0000 7fd0a37fe640 1 -- 172.21.15.92:0/3051063072 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac019450 con 0x560ed402cf00 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 -- 172.21.15.92:0/3051063072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402cf00 msgr2=0x560ed403da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/3051063072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402cf00 0x560ed403da30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009250 tx=0x7fd0ac009d00 comp rx=0 tx=0).stop 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 -- 172.21.15.92:0/3051063072 shutdown_connections 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/3051063072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed4032070 0x560ed402d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/3051063072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed402cf00 0x560ed403da30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/3051063072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402c510 0x560ed4022930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 -- 172.21.15.92:0/3051063072 >> 172.21.15.92:0/3051063072 conn(0x560ed4023670 msgr2=0x560ed4022220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 -- 172.21.15.92:0/3051063072 shutdown_connections 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 -- 172.21.15.92:0/3051063072 wait complete. 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 -- start start 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 Processor -- start 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402c510 0x560ed403ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed403b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed4032070 0x560ed403c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1a7e10 con 0x560ed402c510 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1a7740 con 0x560ed402cf00 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc1a4980 con 0x560ed4032070 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.972+0000 7fd0c6a48740 10 client.0 _ll_register_callbacks cb 0x7fd0b0009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402c510 0x560ed403ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402c510 0x560ed403ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45574/0 (socket says 172.21.15.92:45574) 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 -- 172.21.15.92:0/2532499007 learned_addr learned my addr 172.21.15.92:0/2532499007 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed4032070 0x560ed403c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed403b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 -- 172.21.15.92:0/2532499007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed4032070 msgr2=0x560ed403c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed4032070 0x560ed403c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 -- 172.21.15.92:0/2532499007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 msgr2=0x560ed403b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:55:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed403b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 -- 172.21.15.92:0/2532499007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed403c890 con 0x560ed402c510 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed403b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c13fe640 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402c510 0x560ed403ad50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0012ec0 tx=0x7fd0b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801f7b0b25128125 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2532499007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0004b30 con 0x560ed402c510 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2532499007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ac016070 con 0x560ed402c510 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2532499007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd0b0004cd0 con 0x560ed402c510 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2532499007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0b0031030 con 0x560ed402c510 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c6a48740 10 client.15573 fetch_fsmap learned FSMap version 29 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.973+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2532499007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ed403dc10 con 0x560ed402c510 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2532499007 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac016070 con 0x560ed402c510 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2532499007 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0b0027640 con 0x560ed402c510 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0c6a48740 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:55:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0c6a48740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:55:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0c6a48740 20 client.15573 populate_metadata read hostname 'smithi092' 2024-03-19T01:55:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560ed40a8ba0 0x560ed40aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:55:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0c6a48740 4 client.15573 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:55:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2532499007 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ed40362e0 con 0x560ed40a8ba0 2024-03-19T01:55:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560ed40a8ba0 0x560ed40aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:55:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0c1bff640 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560ed40a8ba0 0x560ed40aafc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b859c1073b7396 server_cookie=8ef8eedcb6323f6f in_seq=0 out_seq=0 2024-03-19T01:55:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.974+0000 7fd0a17fa640 10 client.15573 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0a17fa640 1 -- 172.21.15.92:0/2532499007 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ed40362e0 con 0x560ed40a8ba0 2024-03-19T01:55:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0a17fa640 10 client.15573 handle_command_reply: tid=0 2024-03-19T01:55:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 client.15573 shutdown 2024-03-19T01:55:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2532499007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402c510 msgr2=0x560ed403ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402c510 0x560ed403ad50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0012ec0 tx=0x7fd0b00046c0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2532499007 shutdown_connections 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560ed40a8ba0 0x560ed40aafc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ed4032070 0x560ed403c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ed402cf00 0x560ed403b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0a17fa640 0 client.15573 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 --2- 172.21.15.92:0/2532499007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560ed402c510 0x560ed403ad50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2532499007 >> 172.21.15.92:0/2532499007 conn(0x560ed4023670 msgr2=0x560ed40268d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.981+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2532499007 shutdown_connections 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c6a48740 1 -- 172.21.15.92:0/2532499007 wait complete. 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c6a48740 20 client.15573 trim_cache size 0 max 16384 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:55:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:55:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:55:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 -- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0a40a2920 msgr2=0x7fd0a40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0a40a2920 0x7fd0a40a4de0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd0b404c000 tx=0x7fd0b4034a60 comp rx=0 tx=0).stop 2024-03-19T01:55:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 -- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 msgr2=0x7fd0bc19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:55:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 0x7fd0bc19f040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4037040 tx=0x7fd0b4032f40 comp rx=0 tx=0).stop 2024-03-19T01:55:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 -- 172.21.15.92:0/247462473 shutdown_connections 2024-03-19T01:55:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0bc1022f0 0x7fd0bc19f040 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0bc101930 0x7fd0bc198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd0a40a2920 0x7fd0a40a4de0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 --2- 172.21.15.92:0/247462473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0bc100740 0x7fd0bc197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:55:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 -- 172.21.15.92:0/247462473 >> 172.21.15.92:0/247462473 conn(0x7fd0bc0fbcb0 msgr2=0x7fd0bc104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:55:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 -- 172.21.15.92:0/247462473 shutdown_connections 2024-03-19T01:55:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:55:58.982+0000 7fd0c3e63640 1 -- 172.21.15.92:0/247462473 wait complete. 2024-03-19T01:55:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:55:58 smithi092 ceph-mon[62574]: pgmap v446: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:55:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:55:58 smithi144 ceph-mon[67199]: pgmap v446: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:56:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:00 smithi159 ceph-mon[67110]: pgmap v447: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:56:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:00 smithi092 ceph-mon[62574]: pgmap v447: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:56:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:00 smithi144 ceph-mon[67199]: pgmap v447: 97 pgs: 97 active+clean; 1.3 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:56:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:02 smithi159 ceph-mon[67110]: pgmap v448: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-03-19T01:56:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:02 smithi092 ceph-mon[62574]: pgmap v448: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-03-19T01:56:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:02 smithi144 ceph-mon[67199]: pgmap v448: 97 pgs: 97 active+clean; 1.3 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-03-19T01:56:04.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:03 smithi092 ceph-mon[62574]: pgmap v449: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:04.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:03 smithi144 ceph-mon[67199]: pgmap v449: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:04.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:03 smithi159 ceph-mon[67110]: pgmap v449: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:06.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:06 smithi144 ceph-mon[67199]: pgmap v450: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:06 smithi159 ceph-mon[67110]: pgmap v450: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:06 smithi092 ceph-mon[62574]: pgmap v450: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:08.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:08 smithi144 ceph-mon[67199]: pgmap v451: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:56:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:08 smithi159 ceph-mon[67110]: pgmap v451: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:56:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:08 smithi092 ceph-mon[62574]: pgmap v451: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:56:10.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:10 smithi144 ceph-mon[67199]: pgmap v452: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:56:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:10 smithi159 ceph-mon[67110]: pgmap v452: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:56:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:10 smithi092 ceph-mon[62574]: pgmap v452: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.070+0000 7fce07c12640 1 -- 172.21.15.92:0/4102963484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce001022f0 msgr2=0x7fce00108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.070+0000 7fce07c12640 1 --2- 172.21.15.92:0/4102963484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce001022f0 0x7fce00108b80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009ca0 tx=0x7fcdfc02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.070+0000 7fce07c12640 1 -- 172.21.15.92:0/4102963484 shutdown_connections 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.070+0000 7fce07c12640 1 --2- 172.21.15.92:0/4102963484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce001022f0 0x7fce00108b80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.070+0000 7fce07c12640 1 --2- 172.21.15.92:0/4102963484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce00101930 0x7fce00101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.070+0000 7fce07c12640 1 --2- 172.21.15.92:0/4102963484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce00100740 0x7fce00100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.070+0000 7fce07c12640 1 -- 172.21.15.92:0/4102963484 >> 172.21.15.92:0/4102963484 conn(0x7fce000fbcb0 msgr2=0x7fce000fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.070+0000 7fce07c12640 1 -- 172.21.15.92:0/4102963484 shutdown_connections 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.070+0000 7fce07c12640 1 -- 172.21.15.92:0/4102963484 wait complete. 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce07c12640 1 Processor -- start 2024-03-19T01:56:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce07c12640 1 -- start start 2024-03-19T01:56:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce07c12640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce00100740 0x7fce00069a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce07c12640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce00101930 0x7fce0019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce07c12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce001022f0 0x7fce0019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce07c12640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce0010c3a0 con 0x7fce00101930 2024-03-19T01:56:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce07c12640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce0010c220 con 0x7fce001022f0 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce07c12640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce0010c520 con 0x7fce00100740 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce051ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce00101930 0x7fce0019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce051ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce00101930 0x7fce0019c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55066/0 (socket says 172.21.15.92:55066) 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce051ad640 1 -- 172.21.15.92:0/33200298 learned_addr learned my addr 172.21.15.92:0/33200298 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce059ae640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce00100740 0x7fce00069a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce061af640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce001022f0 0x7fce0019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce051ad640 1 -- 172.21.15.92:0/33200298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce00100740 msgr2=0x7fce00069a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce051ad640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce00100740 0x7fce00069a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce051ad640 1 -- 172.21.15.92:0/33200298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce001022f0 msgr2=0x7fce0019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce051ad640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce001022f0 0x7fce0019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce051ad640 1 -- 172.21.15.92:0/33200298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0019f680 con 0x7fce00101930 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.071+0000 7fce051ad640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce00101930 0x7fce0019c780 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000e9f0 tx=0x7fcdf000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.072+0000 7fcdf6ffd640 1 -- 172.21.15.92:0/33200298 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf000cd80 con 0x7fce00101930 2024-03-19T01:56:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.072+0000 7fcdf6ffd640 1 -- 172.21.15.92:0/33200298 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf0010910 con 0x7fce00101930 2024-03-19T01:56:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.072+0000 7fce07c12640 1 -- 172.21.15.92:0/33200298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0019f970 con 0x7fce00101930 2024-03-19T01:56:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.072+0000 7fcdf6ffd640 1 -- 172.21.15.92:0/33200298 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0018990 con 0x7fce00101930 2024-03-19T01:56:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.072+0000 7fce07c12640 1 -- 172.21.15.92:0/33200298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0019fec0 con 0x7fce00101930 2024-03-19T01:56:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.074+0000 7fce0a800740 1 Processor -- start 2024-03-19T01:56:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.074+0000 7fce0a800740 1 -- start start 2024-03-19T01:56:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.074+0000 7fce0a800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 0x55d7a0cfb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.074+0000 7fce0a800740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.074+0000 7fce0a800740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.074+0000 7fce0a800740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0c5fbe0 con 0x55d7a0d00c60 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce0a800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0c08960 con 0x55d7a0cfb100 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce0a800740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0c5bc80 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce059ae640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce059ae640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:38812/0 (socket says 172.21.15.92:38812) 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce059ae640 1 -- 172.21.15.92:0/2899554273 learned_addr learned my addr 172.21.15.92:0/2899554273 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce061af640 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 0x55d7a0cfb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fcdf6ffd640 1 -- 172.21.15.92:0/33200298 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fcdf0010ac0 con 0x7fce00101930 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce059ae640 1 -- 172.21.15.92:0/2899554273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 msgr2=0x55d7a0cfb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce059ae640 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 0x55d7a0cfb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce051ad640 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce059ae640 1 -- 172.21.15.92:0/2899554273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 msgr2=0x55d7a0d030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce059ae640 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce059ae640 1 -- 172.21.15.92:0/2899554273 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a0bffd20 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce051ad640 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce059ae640 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d005f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009ca0 tx=0x7fcdf8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f11d12a193df3a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fcdf6ffd640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcdec0a28a0 0x7fcdec0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fcdebfff640 1 -- 172.21.15.92:0/2899554273 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8004220 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fcdebfff640 1 -- 172.21.15.92:0/2899554273 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf80043c0 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fcdebfff640 1 -- 172.21.15.92:0/2899554273 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8019930 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fcdf6ffd640 1 -- 172.21.15.92:0/33200298 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcdf00ceda0 con 0x7fce00101930 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.075+0000 7fce061af640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcdec0a28a0 0x7fcdec0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 -- 172.21.15.92:0/2899554273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 msgr2=0x55d7a0d005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce061af640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcdec0a28a0 0x7fcdec0a4d60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc005e30 tx=0x7fcdfc032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d005f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009ca0 tx=0x7fcdf8011810 comp rx=0 tx=0).stop 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 -- 172.21.15.92:0/2899554273 shutdown_connections 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d005f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 --2- 172.21.15.92:0/2899554273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 0x55d7a0cfb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 -- 172.21.15.92:0/2899554273 >> 172.21.15.92:0/2899554273 conn(0x55d7a0cf6670 msgr2=0x55d7a0cf6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 -- 172.21.15.92:0/2899554273 shutdown_connections 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 -- 172.21.15.92:0/2899554273 wait complete. 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.076+0000 7fce0a800740 1 -- start start 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce0a800740 1 Processor -- start 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce0a800740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0cfb100 0x55d7a0d050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce0a800740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce0a800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce0a800740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0d03c10 con 0x55d7a0cfb100 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce0a800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0c1bdb0 con 0x55d7a0d00c60 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce0a800740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d7a0d03d90 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce0a800740 10 client.0 _ll_register_callbacks cb 0x55d7a0c20e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce059ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0cfb100 0x55d7a0d050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce059ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0cfb100 0x55d7a0d050b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55092/0 (socket says 172.21.15.92:55092) 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce059ae640 1 -- 172.21.15.92:0/2873484521 learned_addr learned my addr 172.21.15.92:0/2873484521 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce051ad640 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce061af640 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce051ad640 1 -- 172.21.15.92:0/2873484521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0d00c60 msgr2=0x55d7a0d0c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce051ad640 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce051ad640 1 -- 172.21.15.92:0/2873484521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0cfb100 msgr2=0x55d7a0d050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce051ad640 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0cfb100 0x55d7a0d050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce051ad640 1 -- 172.21.15.92:0/2873484521 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a0c1ac60 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.077+0000 7fce059ae640 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0cfb100 0x55d7a0d050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce051ad640 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d055f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00cc900 tx=0x7fcdf009b160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f7f33819225173 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fcde97fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fcde97fa640 1 -- 172.21.15.92:0/2873484521 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00a3ef0 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fcde97fa640 1 -- 172.21.15.92:0/2873484521 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcdf0018b30 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce0a800740 1 -- 172.21.15.92:0/2873484521 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf80043c0 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fcde97fa640 1 -- 172.21.15.92:0/2873484521 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdf0018d30 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce0a800740 10 client.25471 fetch_fsmap learned FSMap version 29 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce0a800740 1 -- 172.21.15.92:0/2873484521 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7a0974210 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fcde97fa640 1 -- 172.21.15.92:0/2873484521 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf80043c0 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fcde97fa640 1 -- 172.21.15.92:0/2873484521 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcdf009ad70 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce0a800740 10 client.25471 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce0a800740 10 client.25471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce0a800740 20 client.25471 populate_metadata read hostname 'smithi092' 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce0a800740 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d7a0d10ef0 0x55d7a0d13320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce0a800740 4 client.25471 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.078+0000 7fce0a800740 1 -- 172.21.15.92:0/2873484521 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7a0c6d120 con 0x55d7a0d10ef0 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.079+0000 7fce059ae640 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d7a0d10ef0 0x55d7a0d13320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.079+0000 7fcde97fa640 10 client.25471 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.079+0000 7fce059ae640 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d7a0d10ef0 0x55d7a0d13320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2499852d2388358 server_cookie=1fa8d7fe0d560d32 in_seq=0 out_seq=0 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fcde97fa640 1 -- 172.21.15.92:0/2873484521 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d7a0c6d120 con 0x55d7a0d10ef0 2024-03-19T01:56:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fcde97fa640 10 client.25471 handle_command_reply: tid=0 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 client.25471 shutdown 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 -- 172.21.15.92:0/2873484521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 msgr2=0x55d7a0d055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d055f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00cc900 tx=0x7fcdf009b160 comp rx=0 tx=0).stop 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 -- 172.21.15.92:0/2873484521 shutdown_connections 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d7a0d10ef0 0x55d7a0d13320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d055f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fcde97fa640 0 client.25471 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 --2- 172.21.15.92:0/2873484521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0cfb100 0x55d7a0d050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 -- 172.21.15.92:0/2873484521 >> 172.21.15.92:0/2873484521 conn(0x55d7a0cf6670 msgr2=0x55d7a0cf78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 -- 172.21.15.92:0/2873484521 shutdown_connections 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 1 -- 172.21.15.92:0/2873484521 wait complete. 2024-03-19T01:56:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.080+0000 7fce0a800740 20 client.25471 trim_cache size 0 max 16384 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce0a800740 1 Processor -- start 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce0a800740 1 -- start start 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce0a800740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfb100 0x55d7a0cf5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce0a800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d0c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce0a800740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0cfc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce0a800740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce001a8d00 con 0x55d7a0d00c60 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce0a800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce001aa840 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce0a800740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce001a93d0 con 0x55d7a0cfb100 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce061af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0cfc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce059ae640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfb100 0x55d7a0cf5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce061af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0cfc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55104/0 (socket says 172.21.15.92:55104) 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce061af640 1 -- 172.21.15.92:0/98418659 learned_addr learned my addr 172.21.15.92:0/98418659 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce051ad640 1 --2- 172.21.15.92:0/98418659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d0c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce061af640 1 -- 172.21.15.92:0/98418659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfb100 msgr2=0x55d7a0cf5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce061af640 1 --2- 172.21.15.92:0/98418659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfb100 0x55d7a0cf5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce061af640 1 -- 172.21.15.92:0/98418659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfbaf0 msgr2=0x55d7a0d0c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce061af640 1 --2- 172.21.15.92:0/98418659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d0c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce061af640 1 -- 172.21.15.92:0/98418659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a0a40620 con 0x55d7a0d00c60 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.095+0000 7fce061af640 1 --2- 172.21.15.92:0/98418659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0cfc560 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009ca0 tx=0x7fcdfc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157049d4a83d6c91 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fcdeb7fe640 1 -- 172.21.15.92:0/98418659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc039040 con 0x55d7a0d00c60 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fcdeb7fe640 1 -- 172.21.15.92:0/98418659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdfc003e50 con 0x55d7a0d00c60 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fcdeb7fe640 1 -- 172.21.15.92:0/98418659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc04e460 con 0x55d7a0d00c60 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 -- 172.21.15.92:0/98418659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 msgr2=0x55d7a0cfc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 --2- 172.21.15.92:0/98418659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0cfc560 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009ca0 tx=0x7fcdfc041000 comp rx=0 tx=0).stop 2024-03-19T01:56:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 -- 172.21.15.92:0/98418659 shutdown_connections 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 --2- 172.21.15.92:0/98418659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0cfc560 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 --2- 172.21.15.92:0/98418659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d0c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 --2- 172.21.15.92:0/98418659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfb100 0x55d7a0cf5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 -- 172.21.15.92:0/98418659 >> 172.21.15.92:0/98418659 conn(0x55d7a0cf6670 msgr2=0x55d7a0cf5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 -- 172.21.15.92:0/98418659 shutdown_connections 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 -- 172.21.15.92:0/98418659 wait complete. 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.096+0000 7fce0a800740 1 -- start start 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce0a800740 1 Processor -- start 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce0a800740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 0x55d7a0d0a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce0a800740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d0a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce0a800740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce0a800740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce001aa170 con 0x55d7a0d00c60 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce0a800740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce001a9aa0 con 0x55d7a0cfb100 2024-03-19T01:56:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce0a800740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce001a6ce0 con 0x55d7a0cfbaf0 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce0a800740 10 client.0 _ll_register_callbacks cb 0x7fcdf0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce061af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce051ad640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d0a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce061af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55108/0 (socket says 172.21.15.92:55108) 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce061af640 1 -- 172.21.15.92:0/3731398775 learned_addr learned my addr 172.21.15.92:0/3731398775 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce059ae640 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 0x55d7a0d0a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce061af640 1 -- 172.21.15.92:0/3731398775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 msgr2=0x55d7a0d0a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce061af640 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d0a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce061af640 1 -- 172.21.15.92:0/3731398775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 msgr2=0x55d7a0d0a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce061af640 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 0x55d7a0d0a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce061af640 1 -- 172.21.15.92:0/3731398775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a0d0be40 con 0x55d7a0d00c60 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fce061af640 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0b850 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0324c0 tx=0x7fcdfc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea48c66f76982d2f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.097+0000 7fcde97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:56:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fcde97fa640 1 -- 172.21.15.92:0/3731398775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc039070 con 0x55d7a0d00c60 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fcde97fa640 1 -- 172.21.15.92:0/3731398775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcdfc004600 con 0x55d7a0d00c60 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce0a800740 1 -- 172.21.15.92:0/3731398775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdfc003e50 con 0x55d7a0d00c60 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fcde97fa640 1 -- 172.21.15.92:0/3731398775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc066050 con 0x55d7a0d00c60 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce0a800740 10 client.15588 fetch_fsmap learned FSMap version 29 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce0a800740 1 -- 172.21.15.92:0/3731398775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7a0d77800 con 0x55d7a0d00c60 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fcde97fa640 1 -- 172.21.15.92:0/3731398775 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdfc003e50 con 0x55d7a0d00c60 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fcde97fa640 1 -- 172.21.15.92:0/3731398775 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcdfc04e980 con 0x55d7a0d00c60 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce0a800740 10 client.15588 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce0a800740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce0a800740 20 client.15588 populate_metadata read hostname 'smithi092' 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce0a800740 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d7a0d78230 0x55d7a0d7a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce0a800740 4 client.15588 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce0a800740 1 -- 172.21.15.92:0/3731398775 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7a0c67470 con 0x55d7a0d78230 2024-03-19T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.098+0000 7fce059ae640 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d7a0d78230 0x55d7a0d7a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.099+0000 7fcde97fa640 10 client.15588 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.099+0000 7fce059ae640 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d7a0d78230 0x55d7a0d7a650 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9264cf7be753114 server_cookie=66737b48b5796dca in_seq=0 out_seq=0 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fcde97fa640 1 -- 172.21.15.92:0/3731398775 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7a0c67470 con 0x55d7a0d78230 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fcde97fa640 10 client.15588 handle_command_reply: tid=0 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 client.15588 shutdown 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 -- 172.21.15.92:0/3731398775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 msgr2=0x55d7a0d0b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0b850 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0324c0 tx=0x7fcdfc009360 comp rx=0 tx=0).stop 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 -- 172.21.15.92:0/3731398775 shutdown_connections 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d7a0d78230 0x55d7a0d7a650 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d7a0d00c60 0x55d7a0d0b850 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7a0cfbaf0 0x55d7a0d0a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 --2- 172.21.15.92:0/3731398775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7a0cfb100 0x55d7a0d0a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fcde97fa640 0 client.15588 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 -- 172.21.15.92:0/3731398775 >> 172.21.15.92:0/3731398775 conn(0x55d7a0cf6670 msgr2=0x55d7a0cf5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 -- 172.21.15.92:0/3731398775 shutdown_connections 2024-03-19T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 1 -- 172.21.15.92:0/3731398775 wait complete. 2024-03-19T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.105+0000 7fce0a800740 20 client.15588 trim_cache size 0 max 16384 2024-03-19T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 -- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcdec0a28a0 msgr2=0x7fcdec0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcdec0a28a0 0x7fcdec0a4d60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc005e30 tx=0x7fcdfc032590 comp rx=0 tx=0).stop 2024-03-19T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 -- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce00101930 msgr2=0x7fce0019c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce00101930 0x7fce0019c780 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000e9f0 tx=0x7fcdf000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 -- 172.21.15.92:0/33200298 shutdown_connections 2024-03-19T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce001022f0 0x7fce0019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce00101930 0x7fce0019c780 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcdec0a28a0 0x7fcdec0a4d60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 --2- 172.21.15.92:0/33200298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce00100740 0x7fce00069a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 -- 172.21.15.92:0/33200298 >> 172.21.15.92:0/33200298 conn(0x7fce000fbcb0 msgr2=0x7fce00104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 -- 172.21.15.92:0/33200298 shutdown_connections 2024-03-19T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:12.106+0000 7fce07c12640 1 -- 172.21.15.92:0/33200298 wait complete. 2024-03-19T01:56:12.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:12 smithi144 ceph-mon[67199]: pgmap v453: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:56:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:12 smithi159 ceph-mon[67110]: pgmap v453: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:56:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:12 smithi092 ceph-mon[62574]: pgmap v453: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:56:14.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:14 smithi144 ceph-mon[67199]: pgmap v454: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:14.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:14 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:56:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:14 smithi159 ceph-mon[67110]: pgmap v454: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:14 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:56:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:14 smithi092 ceph-mon[62574]: pgmap v454: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:14 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:56:15.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:56:15.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:56:15.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:56:15.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:56:15.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:56:15.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:56:16.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:56:16.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:56:16.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:56:16.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:16 smithi144 ceph-mon[67199]: pgmap v455: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-19T01:56:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:16 smithi159 ceph-mon[67110]: pgmap v455: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-19T01:56:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:16 smithi092 ceph-mon[62574]: pgmap v455: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-19T01:56:18.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:18 smithi144 ceph-mon[67199]: pgmap v456: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:56:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:18 smithi159 ceph-mon[67110]: pgmap v456: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:56:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:18 smithi092 ceph-mon[62574]: pgmap v456: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:56:20.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:20 smithi144 ceph-mon[67199]: pgmap v457: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:56:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:20 smithi159 ceph-mon[67110]: pgmap v457: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:56:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:20 smithi092 ceph-mon[62574]: pgmap v457: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:56:22.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:22 smithi144 ceph-mon[67199]: pgmap v458: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:22 smithi159 ceph-mon[67110]: pgmap v458: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:22 smithi092 ceph-mon[62574]: pgmap v458: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:24.862 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:24 smithi092 ceph-mon[62574]: pgmap v459: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:24 smithi159 ceph-mon[67110]: pgmap v459: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:25.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:24 smithi144 ceph-mon[67199]: pgmap v459: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.193+0000 7f30e0d10640 1 -- 172.21.15.92:0/1302669215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 msgr2=0x7f30dc109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.193+0000 7f30e0d10640 1 --2- 172.21.15.92:0/1302669215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 0x7f30dc109990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009900 tx=0x7f30cc02bc30 comp rx=0 tx=0).stop 2024-03-19T01:56:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.193+0000 7f30e0d10640 1 -- 172.21.15.92:0/1302669215 shutdown_connections 2024-03-19T01:56:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.193+0000 7f30e0d10640 1 --2- 172.21.15.92:0/1302669215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 0x7f30dc109990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.193+0000 7f30e0d10640 1 --2- 172.21.15.92:0/1302669215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30dc104c70 0x7f30dc107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.193+0000 7f30e0d10640 1 --2- 172.21.15.92:0/1302669215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30dc102340 0x7f30dc104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.193+0000 7f30e0d10640 1 -- 172.21.15.92:0/1302669215 >> 172.21.15.92:0/1302669215 conn(0x7f30dc0fbcd0 msgr2=0x7f30dc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.193+0000 7f30e0d10640 1 -- 172.21.15.92:0/1302669215 shutdown_connections 2024-03-19T01:56:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.193+0000 7f30e0d10640 1 -- 172.21.15.92:0/1302669215 wait complete. 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30e0d10640 1 Processor -- start 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30e0d10640 1 -- start start 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30e0d10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30dc102340 0x7f30dc197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30e0d10640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30dc104c70 0x7f30dc197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30e0d10640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 0x7f30dc19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30e0d10640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc10d500 con 0x7f30dc1075a0 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30e0d10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc10d380 con 0x7f30dc102340 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30e0d10640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc10d680 con 0x7f30dc104c70 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30dad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 0x7f30dc19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30dad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 0x7f30dc19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38016/0 (socket says 172.21.15.92:38016) 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30dad9d640 1 -- 172.21.15.92:0/291197496 learned_addr learned my addr 172.21.15.92:0/291197496 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.194+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30dc104c70 0x7f30dc197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30da59c640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30dc102340 0x7f30dc197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30dad9d640 1 -- 172.21.15.92:0/291197496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30dc104c70 msgr2=0x7f30dc197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30dad9d640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30dc104c70 0x7f30dc197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30dad9d640 1 -- 172.21.15.92:0/291197496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30dc102340 msgr2=0x7f30dc197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30dad9d640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30dc102340 0x7f30dc197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30dad9d640 1 -- 172.21.15.92:0/291197496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30dc19f6d0 con 0x7f30dc1075a0 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30dc104c70 0x7f30dc197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30dad9d640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 0x7f30dc19f060 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f30cc005ab0 tx=0x7f30cc02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30cb7fe640 1 -- 172.21.15.92:0/291197496 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc041070 con 0x7f30dc1075a0 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30cb7fe640 1 -- 172.21.15.92:0/291197496 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30cc004450 con 0x7f30dc1075a0 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30e0d10640 1 -- 172.21.15.92:0/291197496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30dc19f960 con 0x7f30dc1075a0 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30cb7fe640 1 -- 172.21.15.92:0/291197496 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc003e60 con 0x7f30dc1075a0 2024-03-19T01:56:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.195+0000 7f30e0d10640 1 -- 172.21.15.92:0/291197496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30dc19fe50 con 0x7f30dc1075a0 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.197+0000 7f30cb7fe640 1 -- 172.21.15.92:0/291197496 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f30cc044070 con 0x7f30dc1075a0 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30e39db740 1 Processor -- start 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30e39db740 1 -- start start 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30e39db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351751d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30e39db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x5635175225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30e39db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 0x5635175250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30e39db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563517481be0 con 0x56351751d100 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30e39db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56351742a960 con 0x563517522c60 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30e39db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56351747dc80 con 0x56351751daf0 2024-03-19T01:56:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30cb7fe640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f30c40a28a0 0x7f30c40a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351751d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351751d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38028/0 (socket says 172.21.15.92:38028) 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30d9d9b640 1 -- 172.21.15.92:0/4152039989 learned_addr learned my addr 172.21.15.92:0/4152039989 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30dad9d640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f30c40a28a0 0x7f30c40a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30da59c640 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x5635175225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 0x5635175250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.198+0000 7f30cb7fe640 1 -- 172.21.15.92:0/291197496 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f30cc0f06c0 con 0x7f30dc1075a0 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30d9d9b640 1 -- 172.21.15.92:0/4152039989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 msgr2=0x5635175225f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x5635175225f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30d9d9b640 1 -- 172.21.15.92:0/4152039989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 msgr2=0x5635175250f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 0x5635175250f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30d9d9b640 1 -- 172.21.15.92:0/4152039989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563517421d20 con 0x56351751d100 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30da59c640 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x5635175225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:56:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351751d520 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f30d0011ba0 tx=0x7f30d0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c132cd2cd7d0792 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30dad9d640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f30c40a28a0 0x7f30c40a4d60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f30cc02c7b0 tx=0x7f30cc08ed40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30c8ff9640 1 -- 172.21.15.92:0/4152039989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d000fc50 con 0x56351751d100 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30c8ff9640 1 -- 172.21.15.92:0/4152039989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d000fdf0 con 0x56351751d100 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.199+0000 7f30c8ff9640 1 -- 172.21.15.92:0/4152039989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d000e3b0 con 0x56351751d100 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 -- 172.21.15.92:0/4152039989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 msgr2=0x56351751d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351751d520 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f30d0011ba0 tx=0x7f30d0009b30 comp rx=0 tx=0).stop 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 -- 172.21.15.92:0/4152039989 shutdown_connections 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 0x5635175250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x5635175225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 --2- 172.21.15.92:0/4152039989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351751d520 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 -- 172.21.15.92:0/4152039989 >> 172.21.15.92:0/4152039989 conn(0x563517518670 msgr2=0x563517518a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 -- 172.21.15.92:0/4152039989 shutdown_connections 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 -- 172.21.15.92:0/4152039989 wait complete. 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 -- start start 2024-03-19T01:56:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 Processor -- start 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x563517527140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751daf0 0x563517527680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563517522c60 0x56351752e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563517525c10 con 0x56351751d100 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56351743ddb0 con 0x56351751daf0 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563517525d90 con 0x563517522c60 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30e39db740 10 client.0 _ll_register_callbacks cb 0x563517442e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x563517527140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.200+0000 7f30d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x563517527140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38044/0 (socket says 172.21.15.92:38044) 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30d9d9b640 1 -- 172.21.15.92:0/1310318586 learned_addr learned my addr 172.21.15.92:0/1310318586 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30da59c640 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751daf0 0x563517527680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563517522c60 0x56351752e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30da59c640 1 -- 172.21.15.92:0/1310318586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563517522c60 msgr2=0x56351752e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30da59c640 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563517522c60 0x56351752e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30da59c640 1 -- 172.21.15.92:0/1310318586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 msgr2=0x563517527140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30da59c640 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x563517527140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30da59c640 1 -- 172.21.15.92:0/1310318586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56351743cc60 con 0x56351751daf0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30da59c640 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751daf0 0x563517527680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x56351751def0 tx=0x7f30d400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c423cc0103d7661 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30c1ffb640 1 -- 172.21.15.92:0/1310318586 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4003970 con 0x56351751daf0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30e39db740 1 -- 172.21.15.92:0/1310318586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d000fdf0 con 0x56351751daf0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30c1ffb640 1 -- 172.21.15.92:0/1310318586 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f30d4003b10 con 0x56351751daf0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30c1ffb640 1 -- 172.21.15.92:0/1310318586 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30d4003d10 con 0x56351751daf0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30e39db740 10 client.25436 fetch_fsmap learned FSMap version 29 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30e39db740 1 -- 172.21.15.92:0/1310318586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563517196210 con 0x56351751daf0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.201+0000 7f30c1ffb640 1 -- 172.21.15.92:0/1310318586 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d000fdf0 con 0x56351751daf0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30c1ffb640 1 -- 172.21.15.92:0/1310318586 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f30d4019030 con 0x56351751daf0 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30e39db740 10 client.25436 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:56:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30e39db740 10 client.25436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30e39db740 20 client.25436 populate_metadata read hostname 'smithi092' 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30e39db740 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563517532f60 0x5635175353d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30e39db740 4 client.25436 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30e39db740 1 -- 172.21.15.92:0/1310318586 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56351748f120 con 0x563517532f60 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563517532f60 0x5635175353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30c1ffb640 10 client.25436 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.202+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563517532f60 0x5635175353d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890841c6d629a2 server_cookie=fec086772879dc7e in_seq=0 out_seq=0 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30c1ffb640 1 -- 172.21.15.92:0/1310318586 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56351748f120 con 0x563517532f60 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30c1ffb640 10 client.25436 handle_command_reply: tid=0 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 client.25436 shutdown 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 -- 172.21.15.92:0/1310318586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751daf0 msgr2=0x563517527680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751daf0 0x563517527680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x56351751def0 tx=0x7f30d400f790 comp rx=0 tx=0).stop 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 -- 172.21.15.92:0/1310318586 shutdown_connections 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563517532f60 0x5635175353d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563517522c60 0x56351752e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751daf0 0x563517527680 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30c1ffb640 0 client.25436 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 --2- 172.21.15.92:0/1310318586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x563517527140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 -- 172.21.15.92:0/1310318586 >> 172.21.15.92:0/1310318586 conn(0x563517518670 msgr2=0x5635175198c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 -- 172.21.15.92:0/1310318586 shutdown_connections 2024-03-19T01:56:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 1 -- 172.21.15.92:0/1310318586 wait complete. 2024-03-19T01:56:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.203+0000 7f30e39db740 20 client.25436 trim_cache size 0 max 16384 2024-03-19T01:56:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30e39db740 1 Processor -- start 2024-03-19T01:56:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30e39db740 1 -- start start 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30e39db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751d100 0x56351751bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30e39db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x56351752e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30e39db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563517522c60 0x563517539c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30e39db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc1a8c40 con 0x563517522c60 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30e39db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc1aa780 con 0x56351751d100 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30e39db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc1a9310 con 0x56351751daf0 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563517522c60 0x563517539c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563517522c60 0x563517539c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38056/0 (socket says 172.21.15.92:38056) 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 -- 172.21.15.92:0/1807020982 learned_addr learned my addr 172.21.15.92:0/1807020982 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30da59c640 1 --2- 172.21.15.92:0/1807020982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x56351752e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/1807020982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751d100 0x56351751bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 -- 172.21.15.92:0/1807020982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 msgr2=0x56351752e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/1807020982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x56351752e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 -- 172.21.15.92:0/1807020982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751d100 msgr2=0x56351751bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/1807020982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751d100 0x56351751bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 -- 172.21.15.92:0/1807020982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563517262620 con 0x563517522c60 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/1807020982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563517522c60 0x563517539c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f30d0000c00 tx=0x7f30d0009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf645d23ceafbda1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30c3fff640 1 -- 172.21.15.92:0/1807020982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d000fdf0 con 0x563517522c60 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30c3fff640 1 -- 172.21.15.92:0/1807020982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d002fbc0 con 0x563517522c60 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.218+0000 7f30c3fff640 1 -- 172.21.15.92:0/1807020982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0005410 con 0x563517522c60 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 -- 172.21.15.92:0/1807020982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563517522c60 msgr2=0x563517539c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 --2- 172.21.15.92:0/1807020982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563517522c60 0x563517539c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f30d0000c00 tx=0x7f30d0009f30 comp rx=0 tx=0).stop 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 -- 172.21.15.92:0/1807020982 shutdown_connections 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 --2- 172.21.15.92:0/1807020982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563517522c60 0x563517539c80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 --2- 172.21.15.92:0/1807020982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x56351752e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 --2- 172.21.15.92:0/1807020982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56351751d100 0x56351751bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 -- 172.21.15.92:0/1807020982 >> 172.21.15.92:0/1807020982 conn(0x563517518670 msgr2=0x56351751b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 -- 172.21.15.92:0/1807020982 shutdown_connections 2024-03-19T01:56:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 -- 172.21.15.92:0/1807020982 wait complete. 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 -- start start 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.219+0000 7f30e39db740 1 Processor -- start 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30e39db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351752e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30e39db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x56351752e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30e39db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 0x56351752c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30e39db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc1aa0b0 con 0x56351751d100 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30e39db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc1a99e0 con 0x563517522c60 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30e39db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30dc1a6c20 con 0x56351751daf0 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30e39db740 10 client.0 _ll_register_callbacks cb 0x7f30d4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351752e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351752e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38072/0 (socket says 172.21.15.92:38072) 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 -- 172.21.15.92:0/4258922877 learned_addr learned my addr 172.21.15.92:0/4258922877 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30da59c640 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x56351752e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 0x56351752c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 -- 172.21.15.92:0/4258922877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 msgr2=0x56351752e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x56351752e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 -- 172.21.15.92:0/4258922877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 msgr2=0x56351752c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 0x56351752c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 -- 172.21.15.92:0/4258922877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56351752d1a0 con 0x56351751d100 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30d9d9b640 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351752e1d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009fe0 tx=0x7f30d0009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbd959c705f269b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30c1ffb640 1 -- 172.21.15.92:0/4258922877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0005390 con 0x56351751d100 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30c1ffb640 1 -- 172.21.15.92:0/4258922877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f30d0005530 con 0x56351751d100 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.220+0000 7f30e39db740 1 -- 172.21.15.92:0/4258922877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d002fbc0 con 0x56351751d100 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30c1ffb640 1 -- 172.21.15.92:0/4258922877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30d0041050 con 0x56351751d100 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30e39db740 10 client.15606 fetch_fsmap learned FSMap version 29 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30e39db740 1 -- 172.21.15.92:0/4258922877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563517599ed0 con 0x56351751d100 2024-03-19T01:56:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30c1ffb640 1 -- 172.21.15.92:0/4258922877 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d002fbc0 con 0x56351751d100 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30c1ffb640 1 -- 172.21.15.92:0/4258922877 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f30d001f3f0 con 0x56351751d100 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30e39db740 10 client.15606 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30e39db740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30e39db740 20 client.15606 populate_metadata read hostname 'smithi092' 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30e39db740 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56351759a6a0 0x56351759cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30e39db740 4 client.15606 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30e39db740 1 -- 172.21.15.92:0/4258922877 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563517489470 con 0x56351759a6a0 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30da59c640 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56351759a6a0 0x56351759cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30c1ffb640 10 client.15606 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.221+0000 7f30da59c640 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56351759a6a0 0x56351759cac0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c8e5082746a5c6 server_cookie=4f66f014a09d3b78 in_seq=0 out_seq=0 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30c1ffb640 1 -- 172.21.15.92:0/4258922877 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563517489470 con 0x56351759a6a0 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30c1ffb640 10 client.15606 handle_command_reply: tid=0 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 client.15606 shutdown 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 -- 172.21.15.92:0/4258922877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 msgr2=0x56351752e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351752e1d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009fe0 tx=0x7f30d0009ea0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 -- 172.21.15.92:0/4258922877 shutdown_connections 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56351759a6a0 0x56351759cac0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563517522c60 0x56351752c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30c1ffb640 0 client.15606 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56351751daf0 0x56351752e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 --2- 172.21.15.92:0/4258922877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56351751d100 0x56351752e1d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 -- 172.21.15.92:0/4258922877 >> 172.21.15.92:0/4258922877 conn(0x563517518670 msgr2=0x56351751c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 -- 172.21.15.92:0/4258922877 shutdown_connections 2024-03-19T01:56:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 1 -- 172.21.15.92:0/4258922877 wait complete. 2024-03-19T01:56:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.228+0000 7f30e39db740 20 client.15606 trim_cache size 0 max 16384 2024-03-19T01:56:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:56:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:56:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:56:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:56:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 -- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f30c40a28a0 msgr2=0x7f30c40a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f30c40a28a0 0x7f30c40a4d60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f30cc02c7b0 tx=0x7f30cc08ed40 comp rx=0 tx=0).stop 2024-03-19T01:56:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 -- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 msgr2=0x7f30dc19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 0x7f30dc19f060 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f30cc005ab0 tx=0x7f30cc02c6c0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 -- 172.21.15.92:0/291197496 shutdown_connections 2024-03-19T01:56:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30dc1075a0 0x7f30dc19f060 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f30c40a28a0 0x7f30c40a4d60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30dc104c70 0x7f30dc197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 --2- 172.21.15.92:0/291197496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30dc102340 0x7f30dc197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 -- 172.21.15.92:0/291197496 >> 172.21.15.92:0/291197496 conn(0x7f30dc0fbcd0 msgr2=0x7f30dc102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 -- 172.21.15.92:0/291197496 shutdown_connections 2024-03-19T01:56:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:25.229+0000 7f30e0d10640 1 -- 172.21.15.92:0/291197496 wait complete. 2024-03-19T01:56:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:26 smithi159 ceph-mon[67110]: pgmap v460: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T01:56:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:26 smithi092 ceph-mon[62574]: pgmap v460: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T01:56:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:26 smithi144 ceph-mon[67199]: pgmap v460: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T01:56:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:28 smithi159 ceph-mon[67110]: pgmap v461: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-03-19T01:56:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:28 smithi092 ceph-mon[62574]: pgmap v461: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-03-19T01:56:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:28 smithi144 ceph-mon[67199]: pgmap v461: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-03-19T01:56:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:30 smithi159 ceph-mon[67110]: pgmap v462: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:56:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:30 smithi092 ceph-mon[62574]: pgmap v462: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:56:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:30 smithi144 ceph-mon[67199]: pgmap v462: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:56:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:32 smithi159 ceph-mon[67110]: pgmap v463: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:32 smithi092 ceph-mon[62574]: pgmap v463: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:32 smithi144 ceph-mon[67199]: pgmap v463: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:34 smithi159 ceph-mon[67110]: pgmap v464: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:34 smithi092 ceph-mon[62574]: pgmap v464: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:34 smithi144 ceph-mon[67199]: pgmap v464: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:56:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:36 smithi159 ceph-mon[67110]: pgmap v465: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:36 smithi092 ceph-mon[62574]: pgmap v465: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:36 smithi144 ceph-mon[67199]: pgmap v465: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:56:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 -- 172.21.15.92:0/1785316308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c1022f0 msgr2=0x7f888c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 --2- 172.21.15.92:0/1785316308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c1022f0 0x7f888c108b80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8884009ca0 tx=0x7f888402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 -- 172.21.15.92:0/1785316308 shutdown_connections 2024-03-19T01:56:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 --2- 172.21.15.92:0/1785316308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c1022f0 0x7f888c108b80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 --2- 172.21.15.92:0/1785316308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f888c101930 0x7f888c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 --2- 172.21.15.92:0/1785316308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c100740 0x7f888c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 -- 172.21.15.92:0/1785316308 >> 172.21.15.92:0/1785316308 conn(0x7f888c0fbcb0 msgr2=0x7f888c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 -- 172.21.15.92:0/1785316308 shutdown_connections 2024-03-19T01:56:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 -- 172.21.15.92:0/1785316308 wait complete. 2024-03-19T01:56:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 Processor -- start 2024-03-19T01:56:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.316+0000 7f8892aa0640 1 -- start start 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f8892aa0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c100740 0x7f888c196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f8892aa0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 0x7f888c196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f8892aa0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f888c1022f0 0x7f888c19ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f8892aa0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f888c10c3a0 con 0x7f888c100740 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f8892aa0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f888c10c220 con 0x7f888c101930 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f8892aa0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f888c10c520 con 0x7f888c1022f0 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889083c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c100740 0x7f888c196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889083c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c100740 0x7f888c196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54094/0 (socket says 172.21.15.92:54094) 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889083c640 1 -- 172.21.15.92:0/3993398775 learned_addr learned my addr 172.21.15.92:0/3993398775 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f888bfff640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 0x7f888c196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889103d640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f888c1022f0 0x7f888c19ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889083c640 1 -- 172.21.15.92:0/3993398775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f888c1022f0 msgr2=0x7f888c19ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889083c640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f888c1022f0 0x7f888c19ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889083c640 1 -- 172.21.15.92:0/3993398775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 msgr2=0x7f888c196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889083c640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 0x7f888c196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889083c640 1 -- 172.21.15.92:0/3993398775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c19e540 con 0x7f888c100740 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f888bfff640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 0x7f888c196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f889083c640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c100740 0x7f888c196920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f888000e9c0 tx=0x7f888000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.317+0000 7f8889ffb640 1 -- 172.21.15.92:0/3993398775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888000cd80 con 0x7f888c100740 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.318+0000 7f8889ffb640 1 -- 172.21.15.92:0/3993398775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88800035f0 con 0x7f888c100740 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.318+0000 7f8892aa0640 1 -- 172.21.15.92:0/3993398775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888c19e830 con 0x7f888c100740 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.318+0000 7f8889ffb640 1 -- 172.21.15.92:0/3993398775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880010690 con 0x7f888c100740 2024-03-19T01:56:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.318+0000 7f8892aa0640 1 -- 172.21.15.92:0/3993398775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888c19edd0 con 0x7f888c100740 2024-03-19T01:56:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.320+0000 7f8889ffb640 1 -- 172.21.15.92:0/3993398775 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f88800037a0 con 0x7f888c100740 2024-03-19T01:56:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.320+0000 7f8895685740 1 Processor -- start 2024-03-19T01:56:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.320+0000 7f8895685740 1 -- start start 2024-03-19T01:56:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.320+0000 7f8895685740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 0x563da7e83730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.320+0000 7f8895685740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 0x563da7e88800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.320+0000 7f8895685740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.320+0000 7f8895685740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563da7de5be0 con 0x563da7e88e70 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.320+0000 7f8895685740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563da7d8e960 con 0x563da7e83d00 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.320+0000 7f8895685740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563da7de1c80 con 0x563da7e83310 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f888bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f889103d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 0x563da7e83730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f888bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54102/0 (socket says 172.21.15.92:54102) 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f888bfff640 1 -- 172.21.15.92:0/3861084473 learned_addr learned my addr 172.21.15.92:0/3861084473 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f889083c640 1 --2- 172.21.15.92:0/3861084473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 0x563da7e88800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f888bfff640 1 -- 172.21.15.92:0/3861084473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 msgr2=0x563da7e83730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f888bfff640 1 --2- 172.21.15.92:0/3861084473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 0x563da7e83730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f888bfff640 1 -- 172.21.15.92:0/3861084473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 msgr2=0x563da7e88800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f888bfff640 1 --2- 172.21.15.92:0/3861084473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 0x563da7e88800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f888bfff640 1 -- 172.21.15.92:0/3861084473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da7d85d20 con 0x563da7e88e70 2024-03-19T01:56:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f8889ffb640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f887c0a2920 0x7f887c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f888bfff640 1 --2- 172.21.15.92:0/3861084473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8b300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f88780114f0 tx=0x7f88780119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461e65dc52ea541d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f889103d640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f887c0a2920 0x7f887c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f8889ffb640 1 -- 172.21.15.92:0/3993398775 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f88800ced80 con 0x7f888c100740 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f8876ffd640 1 -- 172.21.15.92:0/3861084473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887801e070 con 0x563da7e88e70 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f8876ffd640 1 -- 172.21.15.92:0/3861084473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88780035f0 con 0x563da7e88e70 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.321+0000 7f8876ffd640 1 -- 172.21.15.92:0/3861084473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887800f820 con 0x563da7e88e70 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 -- 172.21.15.92:0/3861084473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 msgr2=0x563da7e8b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f889103d640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f887c0a2920 0x7f887c0a4de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8884005e30 tx=0x7f8884032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 --2- 172.21.15.92:0/3861084473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8b300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f88780114f0 tx=0x7f88780119b0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 -- 172.21.15.92:0/3861084473 shutdown_connections 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 --2- 172.21.15.92:0/3861084473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8b300 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 --2- 172.21.15.92:0/3861084473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 0x563da7e88800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 --2- 172.21.15.92:0/3861084473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 0x563da7e83730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 -- 172.21.15.92:0/3861084473 >> 172.21.15.92:0/3861084473 conn(0x563da7e7e880 msgr2=0x563da7e7ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 -- 172.21.15.92:0/3861084473 shutdown_connections 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 -- 172.21.15.92:0/3861084473 wait complete. 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 -- start start 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.322+0000 7f8895685740 1 Processor -- start 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f8895685740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e83310 0x563da7e8d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f8895685740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 0x563da7e8d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f8895685740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e88e70 0x563da7e94840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f8895685740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563da7e8be20 con 0x563da7e83310 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f8895685740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563da7da1db0 con 0x563da7e88e70 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f8895685740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563da7e8bfa0 con 0x563da7e83d00 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f8895685740 10 client.0 _ll_register_callbacks cb 0x563da7da6e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889083c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e83310 0x563da7e8d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889083c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e83310 0x563da7e8d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54104/0 (socket says 172.21.15.92:54104) 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889083c640 1 -- 172.21.15.92:0/2000910432 learned_addr learned my addr 172.21.15.92:0/2000910432 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f888bfff640 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 0x563da7e8d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889103d640 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e88e70 0x563da7e94840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889083c640 1 -- 172.21.15.92:0/2000910432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 msgr2=0x563da7e8d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889083c640 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 0x563da7e8d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889083c640 1 -- 172.21.15.92:0/2000910432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e88e70 msgr2=0x563da7e94840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889083c640 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e88e70 0x563da7e94840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889083c640 1 -- 172.21.15.92:0/2000910432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da7da0c60 con 0x563da7e83310 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f888bfff640 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 0x563da7e8d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f889083c640 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e83310 0x563da7e8d2c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f888000c6a0 tx=0x7f8880097370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86fa310b0ba446a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f886bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f886bfff640 1 -- 172.21.15.92:0/2000910432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888009b090 con 0x563da7e83310 2024-03-19T01:56:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f886bfff640 1 -- 172.21.15.92:0/2000910432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8880003b30 con 0x563da7e83310 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.323+0000 7f8895685740 1 -- 172.21.15.92:0/2000910432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88780035f0 con 0x563da7e83310 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f886bfff640 1 -- 172.21.15.92:0/2000910432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f888009be30 con 0x563da7e83310 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f8895685740 10 client.15618 fetch_fsmap learned FSMap version 29 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f8895685740 1 -- 172.21.15.92:0/2000910432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563da7afa210 con 0x563da7e83310 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f886bfff640 1 -- 172.21.15.92:0/2000910432 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88780035f0 con 0x563da7e83310 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f886bfff640 1 -- 172.21.15.92:0/2000910432 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f888009eb20 con 0x563da7e83310 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f8895685740 10 client.15618 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f8895685740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f8895685740 20 client.15618 populate_metadata read hostname 'smithi092' 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f8895685740 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563da7e99120 0x563da7e9b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f8895685740 4 client.15618 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f8895685740 1 -- 172.21.15.92:0/2000910432 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563da7df3120 con 0x563da7e99120 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.324+0000 7f888bfff640 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563da7e99120 0x563da7e9b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.325+0000 7f886bfff640 10 client.15618 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.325+0000 7f888bfff640 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563da7e99120 0x563da7e9b530 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867e37c3f707c028 server_cookie=408207b4af88e071 in_seq=0 out_seq=0 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.325+0000 7f886bfff640 1 -- 172.21.15.92:0/2000910432 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563da7df3120 con 0x563da7e99120 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.325+0000 7f886bfff640 10 client.15618 handle_command_reply: tid=0 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 client.15618 shutdown 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 -- 172.21.15.92:0/2000910432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e83310 msgr2=0x563da7e8d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e83310 0x563da7e8d2c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f888000c6a0 tx=0x7f8880097370 comp rx=0 tx=0).stop 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 -- 172.21.15.92:0/2000910432 shutdown_connections 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563da7e99120 0x563da7e9b530 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e88e70 0x563da7e94840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 0x563da7e8d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f886bfff640 0 client.15618 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 --2- 172.21.15.92:0/2000910432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e83310 0x563da7e8d2c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 -- 172.21.15.92:0/2000910432 >> 172.21.15.92:0/2000910432 conn(0x563da7e7e880 msgr2=0x563da7e7fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 -- 172.21.15.92:0/2000910432 shutdown_connections 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 1 -- 172.21.15.92:0/2000910432 wait complete. 2024-03-19T01:56:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.326+0000 7f8895685740 20 client.15618 trim_cache size 0 max 16384 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.340+0000 7f8895685740 1 Processor -- start 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.340+0000 7f8895685740 1 -- start start 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f8895685740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 0x563da7e7bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f8895685740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 0x563da7e94800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f8895685740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f8895685740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f888c1a6a40 con 0x563da7e88e70 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f8895685740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f888c1a8580 con 0x563da7e83d00 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f8895685740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f888c1a7110 con 0x563da7e83310 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889103d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889103d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54120/0 (socket says 172.21.15.92:54120) 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889103d640 1 -- 172.21.15.92:0/2455760249 learned_addr learned my addr 172.21.15.92:0/2455760249 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889083c640 1 --2- 172.21.15.92:0/2455760249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 0x563da7e7bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f888bfff640 1 --2- 172.21.15.92:0/2455760249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 0x563da7e94800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889103d640 1 -- 172.21.15.92:0/2455760249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 msgr2=0x563da7e7bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889103d640 1 --2- 172.21.15.92:0/2455760249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 0x563da7e7bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889103d640 1 -- 172.21.15.92:0/2455760249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 msgr2=0x563da7e94800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889103d640 1 --2- 172.21.15.92:0/2455760249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 0x563da7e94800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889103d640 1 -- 172.21.15.92:0/2455760249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da7bc6620 con 0x563da7e88e70 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f889103d640 1 --2- 172.21.15.92:0/2455760249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8d5e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8884036000 tx=0x7f8884041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e235dd70f3f29d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f88767fc640 1 -- 172.21.15.92:0/2455760249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884039040 con 0x563da7e88e70 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f88767fc640 1 -- 172.21.15.92:0/2455760249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8884053070 con 0x563da7e88e70 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.341+0000 7f88767fc640 1 -- 172.21.15.92:0/2455760249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840047a0 con 0x563da7e88e70 2024-03-19T01:56:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 -- 172.21.15.92:0/2455760249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 msgr2=0x563da7e8d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 --2- 172.21.15.92:0/2455760249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8d5e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8884036000 tx=0x7f8884041000 comp rx=0 tx=0).stop 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 -- 172.21.15.92:0/2455760249 shutdown_connections 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 --2- 172.21.15.92:0/2455760249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e8d5e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 --2- 172.21.15.92:0/2455760249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83d00 0x563da7e94800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 --2- 172.21.15.92:0/2455760249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83310 0x563da7e7bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 -- 172.21.15.92:0/2455760249 >> 172.21.15.92:0/2455760249 conn(0x563da7e7e880 msgr2=0x563da7e8ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 -- 172.21.15.92:0/2455760249 shutdown_connections 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 -- 172.21.15.92:0/2455760249 wait complete. 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 -- start start 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.342+0000 7f8895685740 1 Processor -- start 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f8895685740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83310 0x563da7e929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f8895685740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 0x563da7e92ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f8895685740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e93d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f8895685740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f888c1a7eb0 con 0x563da7e88e70 2024-03-19T01:56:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f8895685740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f888c1a77e0 con 0x563da7e83310 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f8895685740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f888c1a4a20 con 0x563da7e83d00 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f8895685740 10 client.0 _ll_register_callbacks cb 0x7f88800d2550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889103d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e93d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889103d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e93d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54128/0 (socket says 172.21.15.92:54128) 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889103d640 1 -- 172.21.15.92:0/1434925769 learned_addr learned my addr 172.21.15.92:0/1434925769 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889083c640 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83310 0x563da7e929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f888bfff640 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 0x563da7e92ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889103d640 1 -- 172.21.15.92:0/1434925769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 msgr2=0x563da7e92ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889103d640 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 0x563da7e92ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889103d640 1 -- 172.21.15.92:0/1434925769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83310 msgr2=0x563da7e929b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889103d640 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83310 0x563da7e929b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889103d640 1 -- 172.21.15.92:0/1434925769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da7e943f0 con 0x563da7e88e70 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f889103d640 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e93d00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8884032a60 tx=0x7f8884009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d839be828d314781 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f886bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f886bfff640 1 -- 172.21.15.92:0/1434925769 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888405b070 con 0x563da7e88e70 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f886bfff640 1 -- 172.21.15.92:0/1434925769 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8884004520 con 0x563da7e88e70 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.343+0000 7f8895685740 1 -- 172.21.15.92:0/1434925769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8884053070 con 0x563da7e88e70 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f886bfff640 1 -- 172.21.15.92:0/1434925769 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8884067050 con 0x563da7e88e70 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f8895685740 10 client.15624 fetch_fsmap learned FSMap version 29 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f8895685740 1 -- 172.21.15.92:0/1434925769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563da7effe00 con 0x563da7e88e70 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f886bfff640 1 -- 172.21.15.92:0/1434925769 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8884053070 con 0x563da7e88e70 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f886bfff640 1 -- 172.21.15.92:0/1434925769 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f888404e790 con 0x563da7e88e70 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f8895685740 10 client.15624 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f8895685740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f8895685740 20 client.15624 populate_metadata read hostname 'smithi092' 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f8895685740 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563da7f007b0 0x563da7f02bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f8895685740 4 client.15624 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:56:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f8895685740 1 -- 172.21.15.92:0/1434925769 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563da7ded470 con 0x563da7f007b0 2024-03-19T01:56:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f889083c640 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563da7f007b0 0x563da7f02bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f889083c640 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563da7f007b0 0x563da7f02bd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5802d6b1e4b08654 server_cookie=6f9abe7b7235fcec in_seq=0 out_seq=0 2024-03-19T01:56:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.344+0000 7f886bfff640 10 client.15624 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f886bfff640 1 -- 172.21.15.92:0/1434925769 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563da7ded470 con 0x563da7f007b0 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f886bfff640 10 client.15624 handle_command_reply: tid=0 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 client.15624 shutdown 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 -- 172.21.15.92:0/1434925769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 msgr2=0x563da7e93d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e93d00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8884032a60 tx=0x7f8884009360 comp rx=0 tx=0).stop 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 -- 172.21.15.92:0/1434925769 shutdown_connections 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563da7f007b0 0x563da7f02bd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563da7e88e70 0x563da7e93d00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563da7e83d00 0x563da7e92ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f886bfff640 0 client.15624 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 --2- 172.21.15.92:0/1434925769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563da7e83310 0x563da7e929b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 -- 172.21.15.92:0/1434925769 >> 172.21.15.92:0/1434925769 conn(0x563da7e7e880 msgr2=0x563da7e7b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 -- 172.21.15.92:0/1434925769 shutdown_connections 2024-03-19T01:56:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 1 -- 172.21.15.92:0/1434925769 wait complete. 2024-03-19T01:56:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.351+0000 7f8895685740 20 client.15624 trim_cache size 0 max 16384 2024-03-19T01:56:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:56:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:56:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:56:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:56:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 -- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f887c0a2920 msgr2=0x7f887c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f887c0a2920 0x7f887c0a4de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8884005e30 tx=0x7f8884032590 comp rx=0 tx=0).stop 2024-03-19T01:56:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 -- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c100740 msgr2=0x7f888c196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c100740 0x7f888c196920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f888000e9c0 tx=0x7f888000ee80 comp rx=0 tx=0).stop 2024-03-19T01:56:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 -- 172.21.15.92:0/3993398775 shutdown_connections 2024-03-19T01:56:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f887c0a2920 0x7f887c0a4de0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f888c1022f0 0x7f888c19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f888c101930 0x7f888c196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 --2- 172.21.15.92:0/3993398775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f888c100740 0x7f888c196920 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 -- 172.21.15.92:0/3993398775 >> 172.21.15.92:0/3993398775 conn(0x7f888c0fbcb0 msgr2=0x7f888c104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 -- 172.21.15.92:0/3993398775 shutdown_connections 2024-03-19T01:56:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:38.352+0000 7f8892aa0640 1 -- 172.21.15.92:0/3993398775 wait complete. 2024-03-19T01:56:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:38 smithi159 ceph-mon[67110]: pgmap v466: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:56:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:38 smithi092 ceph-mon[62574]: pgmap v466: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:56:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:38 smithi144 ceph-mon[67199]: pgmap v466: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:56:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:40 smithi159 ceph-mon[67110]: pgmap v467: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:56:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:40 smithi092 ceph-mon[62574]: pgmap v467: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:56:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:40 smithi144 ceph-mon[67199]: pgmap v467: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:56:42.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:42 smithi144 ceph-mon[67199]: pgmap v468: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T01:56:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:42 smithi159 ceph-mon[67110]: pgmap v468: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T01:56:43.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:42 smithi092 ceph-mon[62574]: pgmap v468: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T01:56:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:44 smithi159 ceph-mon[67110]: pgmap v469: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:56:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:44 smithi092 ceph-mon[62574]: pgmap v469: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:56:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:44 smithi144 ceph-mon[67199]: pgmap v469: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:56:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:46 smithi159 ceph-mon[67110]: pgmap v470: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:56:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:46 smithi092 ceph-mon[62574]: pgmap v470: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:56:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:46 smithi144 ceph-mon[67199]: pgmap v470: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:127/600: open fd = 3 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:128/600: open fd = 3 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.249 INFO:tasks.workunit.client.0.smithi092.stdout:129/600: open fd = 3 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:130/600: open fd = 3 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:131/600: open fd = 3 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.250 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:132/600: open fd = 3 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:133/600: open fd = 3 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:134/600: open fd = 3 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.251 INFO:tasks.workunit.client.0.smithi092.stdout:135/600: open fd = 3 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:136/600: open fd = 3 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:137/600: open fd = 3 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:138/600: open fd = 3 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.252 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:139/600: open fd = 3 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:140/600: open fd = 3 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:141/600: open fd = 3 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:142/600: open fd = 3 2024-03-19T01:56:48.253 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:143/600: open fd = 3 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:144/600: open fd = 3 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.254 INFO:tasks.workunit.client.0.smithi092.stdout:145/600: open fd = 3 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:146/600: open fd = 3 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:147/600: open fd = 3 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.255 INFO:tasks.workunit.client.0.smithi092.stdout:148/600: open fd = 3 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:149/600: open fd = 3 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:150/600: open fd = 3 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:151/600: open fd = 3 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.256 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:152/600: open fd = 3 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:153/600: open fd = 3 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:154/600: open fd = 3 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.257 INFO:tasks.workunit.client.0.smithi092.stdout:155/600: open fd = 3 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:156/600: open fd = 3 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:157/600: open fd = 3 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.258 INFO:tasks.workunit.client.0.smithi092.stdout:158/600: open fd = 3 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:159/600: open fd = 3 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:160/600: open fd = 3 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.259 INFO:tasks.workunit.client.0.smithi092.stdout:161/600: open fd = 3 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:162/600: open fd = 3 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:163/600: open fd = 3 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:164/600: open fd = 3 2024-03-19T01:56:48.260 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:165/600: open fd = 3 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:166/600: open fd = 3 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:167/600: open fd = 3 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.261 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:168/600: open fd = 3 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:169/600: open fd = 3 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:170/600: open fd = 3 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.262 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:171/600: open fd = 3 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:172/600: open fd = 3 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:173/600: open fd = 3 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.263 INFO:tasks.workunit.client.0.smithi092.stdout:174/600: open fd = 3 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:175/600: open fd = 3 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:176/600: open fd = 3 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:177/600: open fd = 3 2024-03-19T01:56:48.264 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:178/600: open fd = 3 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:179/600: open fd = 3 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:180/600: open fd = 3 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.265 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:181/600: open fd = 3 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:182/600: open fd = 3 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:183/600: open fd = 3 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.266 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:184/600: open fd = 3 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:185/600: open fd = 3 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:186/600: open fd = 3 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.267 INFO:tasks.workunit.client.0.smithi092.stdout:187/600: open fd = 3 2024-03-19T01:56:48.268 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.268 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.268 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:56:48.268 INFO:tasks.workunit.client.0.smithi092.stdout:188/600: open fd = 3 2024-03-19T01:56:48.268 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.268 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:56:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:48 smithi159 ceph-mon[67110]: pgmap v471: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:56:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:48 smithi092 ceph-mon[62574]: pgmap v471: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:56:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:48 smithi144 ceph-mon[67199]: pgmap v471: 97 pgs: 97 active+clean; 1.5 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:56:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:50 smithi159 ceph-mon[67110]: pgmap v472: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:56:51.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:50 smithi092 ceph-mon[62574]: pgmap v472: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:56:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:50 smithi144 ceph-mon[67199]: pgmap v472: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:56:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 -- 172.21.15.92:0/822194553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c100740 msgr2=0x7fab1c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 --2- 172.21.15.92:0/822194553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c100740 0x7fab1c100b60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fab10009900 tx=0x7fab1002bc30 comp rx=0 tx=0).stop 2024-03-19T01:56:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 -- 172.21.15.92:0/822194553 shutdown_connections 2024-03-19T01:56:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 --2- 172.21.15.92:0/822194553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c1022f0 0x7fab1c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 --2- 172.21.15.92:0/822194553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab1c101930 0x7fab1c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 --2- 172.21.15.92:0/822194553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c100740 0x7fab1c100b60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 -- 172.21.15.92:0/822194553 >> 172.21.15.92:0/822194553 conn(0x7fab1c0fbcb0 msgr2=0x7fab1c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 -- 172.21.15.92:0/822194553 shutdown_connections 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 -- 172.21.15.92:0/822194553 wait complete. 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 Processor -- start 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 -- start start 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab1c100740 0x7fab1c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c101930 0x7fab1c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c1022f0 0x7fab1c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c0694b0 con 0x7fab1c101930 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c069330 con 0x7fab1c1022f0 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab23220640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c069630 con 0x7fab1c100740 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.440+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c101930 0x7fab1c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c101930 0x7fab1c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36340/0 (socket says 172.21.15.92:36340) 2024-03-19T01:56:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab1bfff640 1 -- 172.21.15.92:0/1343540483 learned_addr learned my addr 172.21.15.92:0/1343540483 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab20fbc640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab1c100740 0x7fab1c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab217bd640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c1022f0 0x7fab1c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab1bfff640 1 -- 172.21.15.92:0/1343540483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab1c100740 msgr2=0x7fab1c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab1bfff640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab1c100740 0x7fab1c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab1bfff640 1 -- 172.21.15.92:0/1343540483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c1022f0 msgr2=0x7fab1c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab1bfff640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c1022f0 0x7fab1c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab1bfff640 1 -- 172.21.15.92:0/1343540483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c19f7b0 con 0x7fab1c101930 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab1bfff640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c101930 0x7fab1c1980d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fab0800e9f0 tx=0x7fab0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab19ffb640 1 -- 172.21.15.92:0/1343540483 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0800cd80 con 0x7fab1c101930 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab19ffb640 1 -- 172.21.15.92:0/1343540483 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab08005500 con 0x7fab1c101930 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab19ffb640 1 -- 172.21.15.92:0/1343540483 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08010690 con 0x7fab1c101930 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab23220640 1 -- 172.21.15.92:0/1343540483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c19faa0 con 0x7fab1c101930 2024-03-19T01:56:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.441+0000 7fab23220640 1 -- 172.21.15.92:0/1343540483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c19fff0 con 0x7fab1c101930 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.443+0000 7fab19ffb640 1 -- 172.21.15.92:0/1343540483 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fab080046c0 con 0x7fab1c101930 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab25e0d740 1 Processor -- start 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab25e0d740 1 -- start start 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f35859f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f3585a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 0x55f3585a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab25e0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f358503be0 con 0x55f35859faf0 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab25e0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f3584ac960 con 0x55f3585a4c60 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab25e0d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f3584ffc80 con 0x55f35859f100 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab19ffb640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fab0c0a2920 0x7fab0c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab20fbc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f3585a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab217bd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f35859f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab20fbc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f3585a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36356/0 (socket says 172.21.15.92:36356) 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab20fbc640 1 -- 172.21.15.92:0/1946633089 learned_addr learned my addr 172.21.15.92:0/1946633089 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab217bd640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fab0c0a2920 0x7fab0c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.444+0000 7fab1bfff640 1 --2- 172.21.15.92:0/1946633089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 0x55f3585a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab19ffb640 1 -- 172.21.15.92:0/1343540483 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fab080cedb0 con 0x7fab1c101930 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab217bd640 1 -- 172.21.15.92:0/1946633089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 msgr2=0x55f3585a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab217bd640 1 --2- 172.21.15.92:0/1946633089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 0x55f3585a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab217bd640 1 -- 172.21.15.92:0/1946633089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 msgr2=0x55f3585a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab217bd640 1 --2- 172.21.15.92:0/1946633089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f3585a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab217bd640 1 -- 172.21.15.92:0/1946633089 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3584a3d20 con 0x55f35859f100 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab20fbc640 1 --2- 172.21.15.92:0/1946633089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f3585a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab217bd640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fab0c0a2920 0x7fab0c0a4de0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab1400fc90 tx=0x7fab1400e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab217bd640 1 --2- 172.21.15.92:0/1946633089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f35859f520 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fab14010d00 tx=0x7fab1400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c128a551a25bd4ae server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab06ffd640 1 -- 172.21.15.92:0/1946633089 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14022070 con 0x55f35859f100 2024-03-19T01:56:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab06ffd640 1 -- 172.21.15.92:0/1946633089 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab140104b0 con 0x55f35859f100 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.445+0000 7fab06ffd640 1 -- 172.21.15.92:0/1946633089 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14019040 con 0x55f35859f100 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 -- 172.21.15.92:0/1946633089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 msgr2=0x55f35859f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1946633089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f35859f520 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fab14010d00 tx=0x7fab1400f790 comp rx=0 tx=0).stop 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 -- 172.21.15.92:0/1946633089 shutdown_connections 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1946633089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 0x55f3585a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1946633089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f3585a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1946633089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f35859f520 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 -- 172.21.15.92:0/1946633089 >> 172.21.15.92:0/1946633089 conn(0x55f35859a670 msgr2=0x55f35859aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 -- 172.21.15.92:0/1946633089 shutdown_connections 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 -- 172.21.15.92:0/1946633089 wait complete. 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 -- start start 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.446+0000 7fab25e0d740 1 Processor -- start 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f3585a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859faf0 0x55f3585a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab25e0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f3585a7bb0 con 0x55f3585a4c60 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab25e0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f3584bfdb0 con 0x55f35859faf0 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab25e0d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f3585a7d30 con 0x55f35859f100 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab25e0d740 10 client.0 _ll_register_callbacks cb 0x55f3584c4e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab217bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab20fbc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f3585a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859faf0 0x55f3585a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab217bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36368/0 (socket says 172.21.15.92:36368) 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab217bd640 1 -- 172.21.15.92:0/1832457624 learned_addr learned my addr 172.21.15.92:0/1832457624 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab217bd640 1 -- 172.21.15.92:0/1832457624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 msgr2=0x55f3585a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab217bd640 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f3585a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab217bd640 1 -- 172.21.15.92:0/1832457624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859faf0 msgr2=0x55f3585a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab217bd640 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859faf0 0x55f3585a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab217bd640 1 -- 172.21.15.92:0/1832457624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3584bec60 con 0x55f3585a4c60 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab217bd640 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585b0620 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fab14009540 tx=0x7fab14027af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4df10c78e144a99 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7faafbfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7faafbfff640 1 -- 172.21.15.92:0/1832457624 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14022040 con 0x55f3585a4c60 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7faafbfff640 1 -- 172.21.15.92:0/1832457624 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fab14004960 con 0x55f3585a4c60 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.447+0000 7fab25e0d740 1 -- 172.21.15.92:0/1832457624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab140104b0 con 0x55f3585a4c60 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7faafbfff640 1 -- 172.21.15.92:0/1832457624 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab14004ba0 con 0x55f3585a4c60 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab25e0d740 10 client.15636 fetch_fsmap learned FSMap version 29 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab25e0d740 1 -- 172.21.15.92:0/1832457624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f358218210 con 0x55f3585a4c60 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7faafbfff640 1 -- 172.21.15.92:0/1832457624 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab140104b0 con 0x55f3585a4c60 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7faafbfff640 1 -- 172.21.15.92:0/1832457624 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fab1402e020 con 0x55f3585a4c60 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab25e0d740 10 client.15636 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab25e0d740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab25e0d740 20 client.15636 populate_metadata read hostname 'smithi092' 2024-03-19T01:56:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55f3585b4e00 0x55f3585b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab25e0d740 4 client.15636 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab25e0d740 1 -- 172.21.15.92:0/1832457624 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f358511120 con 0x55f3585b4e00 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab20fbc640 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55f3585b4e00 0x55f3585b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7faafbfff640 10 client.15636 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.448+0000 7fab20fbc640 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55f3585b4e00 0x55f3585b71f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa06940103a606e2 server_cookie=128e559e32f4ee5b in_seq=0 out_seq=0 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7faafbfff640 1 -- 172.21.15.92:0/1832457624 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f358511120 con 0x55f3585b4e00 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7faafbfff640 10 client.15636 handle_command_reply: tid=0 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 client.15636 shutdown 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 -- 172.21.15.92:0/1832457624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 msgr2=0x55f3585b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585b0620 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fab14009540 tx=0x7fab14027af0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 -- 172.21.15.92:0/1832457624 shutdown_connections 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55f3585b4e00 0x55f3585b71f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585b0620 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859faf0 0x55f3585a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 --2- 172.21.15.92:0/1832457624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f3585a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 -- 172.21.15.92:0/1832457624 >> 172.21.15.92:0/1832457624 conn(0x55f35859a670 msgr2=0x55f35859b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7faafbfff640 0 client.15636 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 -- 172.21.15.92:0/1832457624 shutdown_connections 2024-03-19T01:56:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.449+0000 7fab25e0d740 1 -- 172.21.15.92:0/1832457624 wait complete. 2024-03-19T01:56:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.450+0000 7fab25e0d740 20 client.15636 trim_cache size 0 max 16384 2024-03-19T01:56:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab25e0d740 1 Processor -- start 2024-03-19T01:56:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab25e0d740 1 -- start start 2024-03-19T01:56:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859f100 0x55f3585a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859faf0 0x55f358599930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab25e0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1a8cc0 con 0x55f3585a4c60 2024-03-19T01:56:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab25e0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1aa800 con 0x55f35859f100 2024-03-19T01:56:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab25e0d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1a9390 con 0x55f35859faf0 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab217bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab217bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36374/0 (socket says 172.21.15.92:36374) 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab217bd640 1 -- 172.21.15.92:0/2270287972 learned_addr learned my addr 172.21.15.92:0/2270287972 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab1bfff640 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859faf0 0x55f358599930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab20fbc640 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859f100 0x55f3585a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab217bd640 1 -- 172.21.15.92:0/2270287972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859faf0 msgr2=0x55f358599930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab217bd640 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859faf0 0x55f358599930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab217bd640 1 -- 172.21.15.92:0/2270287972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859f100 msgr2=0x55f3585a86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab217bd640 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859f100 0x55f3585a86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab217bd640 1 -- 172.21.15.92:0/2270287972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3582e4620 con 0x55f3585a4c60 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab20fbc640 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859f100 0x55f3585a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.472+0000 7fab217bd640 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585a0590 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fab14010d00 tx=0x7fab14009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0799b603dabcbf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab067fc640 1 -- 172.21.15.92:0/2270287972 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14022040 con 0x55f3585a4c60 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab067fc640 1 -- 172.21.15.92:0/2270287972 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1403f070 con 0x55f3585a4c60 2024-03-19T01:56:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab067fc640 1 -- 172.21.15.92:0/2270287972 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14030ad0 con 0x55f3585a4c60 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab25e0d740 1 -- 172.21.15.92:0/2270287972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 msgr2=0x55f3585a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab25e0d740 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585a0590 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fab14010d00 tx=0x7fab14009eb0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab25e0d740 1 -- 172.21.15.92:0/2270287972 shutdown_connections 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab25e0d740 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3585a4c60 0x55f3585a0590 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab25e0d740 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859faf0 0x55f358599930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab25e0d740 1 --2- 172.21.15.92:0/2270287972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f35859f100 0x55f3585a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab25e0d740 1 -- 172.21.15.92:0/2270287972 >> 172.21.15.92:0/2270287972 conn(0x55f35859a670 msgr2=0x55f358599220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab25e0d740 1 -- 172.21.15.92:0/2270287972 shutdown_connections 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.473+0000 7fab25e0d740 1 -- 172.21.15.92:0/2270287972 wait complete. 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab25e0d740 1 -- start start 2024-03-19T01:56:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab25e0d740 1 Processor -- start 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f358616550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f358616a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab25e0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 0x55f35861dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab25e0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1aa130 con 0x55f35859faf0 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab25e0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1a9a60 con 0x55f3585a4c60 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab25e0d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1a6ca0 con 0x55f35859f100 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab25e0d740 10 client.0 _ll_register_callbacks cb 0x7fab08005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f358616a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab20fbc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f358616550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab1bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f358616a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36380/0 (socket says 172.21.15.92:36380) 2024-03-19T01:56:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab1bfff640 1 -- 172.21.15.92:0/101700392 learned_addr learned my addr 172.21.15.92:0/101700392 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.474+0000 7fab217bd640 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 0x55f35861dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab1bfff640 1 -- 172.21.15.92:0/101700392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 msgr2=0x55f358616550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab1bfff640 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f358616550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab1bfff640 1 -- 172.21.15.92:0/101700392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 msgr2=0x55f35861dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab1bfff640 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 0x55f35861dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab1bfff640 1 -- 172.21.15.92:0/101700392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f35861e0c0 con 0x55f35859faf0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab1bfff640 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f358616a90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fab0800c6a0 tx=0x7fab0806a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947a7c879923b983 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7faafbfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7faafbfff640 1 -- 172.21.15.92:0/101700392 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab080b4060 con 0x55f35859faf0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7faafbfff640 1 -- 172.21.15.92:0/101700392 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fab08097dd0 con 0x55f35859faf0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab25e0d740 1 -- 172.21.15.92:0/101700392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1403f070 con 0x55f35859faf0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7faafbfff640 1 -- 172.21.15.92:0/101700392 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab0809b760 con 0x55f35859faf0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab25e0d740 10 client.15642 fetch_fsmap learned FSMap version 29 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab25e0d740 1 -- 172.21.15.92:0/101700392 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f35861f4a0 con 0x55f35859faf0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7faafbfff640 1 -- 172.21.15.92:0/101700392 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1403f070 con 0x55f35859faf0 2024-03-19T01:56:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7faafbfff640 1 -- 172.21.15.92:0/101700392 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fab080a3a70 con 0x55f35859faf0 2024-03-19T01:56:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab25e0d740 10 client.15642 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:56:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab25e0d740 10 client.15642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:56:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab25e0d740 20 client.15642 populate_metadata read hostname 'smithi092' 2024-03-19T01:56:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab25e0d740 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55f35861fe80 0x55f3586222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:56:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab25e0d740 4 client.15642 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:56:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.475+0000 7fab25e0d740 1 -- 172.21.15.92:0/101700392 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3585a8e90 con 0x55f35861fe80 2024-03-19T01:56:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.476+0000 7fab20fbc640 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55f35861fe80 0x55f3586222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:56:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.476+0000 7faafbfff640 10 client.15642 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.476+0000 7fab20fbc640 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55f35861fe80 0x55f3586222a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d6374fd83eb2b server_cookie=f2b51a88d4f191b2 in_seq=0 out_seq=0 2024-03-19T01:56:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7faafbfff640 1 -- 172.21.15.92:0/101700392 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3585a8e90 con 0x55f35861fe80 2024-03-19T01:56:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7faafbfff640 10 client.15642 handle_command_reply: tid=0 2024-03-19T01:56:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 client.15642 shutdown 2024-03-19T01:56:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 -- 172.21.15.92:0/101700392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 msgr2=0x55f358616a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f358616a90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fab0800c6a0 tx=0x7fab0806a680 comp rx=0 tx=0).stop 2024-03-19T01:56:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 -- 172.21.15.92:0/101700392 shutdown_connections 2024-03-19T01:56:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55f35861fe80 0x55f3586222a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3585a4c60 0x55f35861dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f35859faf0 0x55f358616a90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7faafbfff640 0 client.15642 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 --2- 172.21.15.92:0/101700392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f35859f100 0x55f358616550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 -- 172.21.15.92:0/101700392 >> 172.21.15.92:0/101700392 conn(0x55f35859a670 msgr2=0x55f358599220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 -- 172.21.15.92:0/101700392 shutdown_connections 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.482+0000 7fab25e0d740 1 -- 172.21.15.92:0/101700392 wait complete. 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab25e0d740 20 client.15642 trim_cache size 0 max 16384 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:56:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:56:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 -- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fab0c0a2920 msgr2=0x7fab0c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fab0c0a2920 0x7fab0c0a4de0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab1400fc90 tx=0x7fab1400e000 comp rx=0 tx=0).stop 2024-03-19T01:56:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 -- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c101930 msgr2=0x7fab1c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:56:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c101930 0x7fab1c1980d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fab0800e9f0 tx=0x7fab0800edf0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 -- 172.21.15.92:0/1343540483 shutdown_connections 2024-03-19T01:56:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c1022f0 0x7fab1c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab1c101930 0x7fab1c1980d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fab0c0a2920 0x7fab0c0a4de0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 --2- 172.21.15.92:0/1343540483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab1c100740 0x7fab1c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:56:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 -- 172.21.15.92:0/1343540483 >> 172.21.15.92:0/1343540483 conn(0x7fab1c0fbcb0 msgr2=0x7fab1c104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:56:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 -- 172.21.15.92:0/1343540483 shutdown_connections 2024-03-19T01:56:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:56:51.483+0000 7fab23220640 1 -- 172.21.15.92:0/1343540483 wait complete. 2024-03-19T01:56:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:52 smithi159 ceph-mon[67110]: pgmap v473: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-03-19T01:56:53.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:52 smithi092 ceph-mon[62574]: pgmap v473: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-03-19T01:56:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:52 smithi144 ceph-mon[67199]: pgmap v473: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 KiB/s wr, 15 op/s 2024-03-19T01:56:54.898 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:54 smithi144 ceph-mon[67199]: pgmap v474: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T01:56:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:54 smithi159 ceph-mon[67110]: pgmap v474: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T01:56:55.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:54 smithi092 ceph-mon[62574]: pgmap v474: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T01:56:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:56 smithi159 ceph-mon[67110]: pgmap v475: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:56:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:56 smithi092 ceph-mon[62574]: pgmap v475: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:56:57.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:56 smithi144 ceph-mon[67199]: pgmap v475: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:56:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:56:58 smithi159 ceph-mon[67110]: pgmap v476: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:56:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:56:58 smithi092 ceph-mon[62574]: pgmap v476: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:56:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:56:58 smithi144 ceph-mon[67199]: pgmap v476: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:57:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:00 smithi159 ceph-mon[67110]: pgmap v477: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T01:57:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:00 smithi092 ceph-mon[62574]: pgmap v477: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T01:57:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:00 smithi144 ceph-mon[67199]: pgmap v477: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T01:57:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:02 smithi159 ceph-mon[67110]: pgmap v478: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-19T01:57:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:02 smithi092 ceph-mon[62574]: pgmap v478: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-19T01:57:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:02 smithi144 ceph-mon[67199]: pgmap v478: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-19T01:57:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.569+0000 7ff5a4697640 1 -- 172.21.15.92:0/2382499674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 msgr2=0x7ff59c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.569+0000 7ff5a4697640 1 --2- 172.21.15.92:0/2382499674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 0x7ff59c108b80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff594009ca0 tx=0x7ff59402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 -- 172.21.15.92:0/2382499674 shutdown_connections 2024-03-19T01:57:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 --2- 172.21.15.92:0/2382499674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 0x7ff59c108b80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 --2- 172.21.15.92:0/2382499674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff59c101930 0x7ff59c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 --2- 172.21.15.92:0/2382499674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c100740 0x7ff59c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 -- 172.21.15.92:0/2382499674 >> 172.21.15.92:0/2382499674 conn(0x7ff59c0fbcb0 msgr2=0x7ff59c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 -- 172.21.15.92:0/2382499674 shutdown_connections 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 -- 172.21.15.92:0/2382499674 wait complete. 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 Processor -- start 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 -- start start 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c100740 0x7ff59c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff59c101930 0x7ff59c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 0x7ff59c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c10c3c0 con 0x7ff59c1022f0 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c10c240 con 0x7ff59c100740 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a4697640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c10c540 con 0x7ff59c101930 2024-03-19T01:57:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a2c34640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 0x7ff59c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a2c34640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 0x7ff59c19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45854/0 (socket says 172.21.15.92:45854) 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a1c32640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff59c101930 0x7ff59c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a2c34640 1 -- 172.21.15.92:0/1005563075 learned_addr learned my addr 172.21.15.92:0/1005563075 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.570+0000 7ff5a2433640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c100740 0x7ff59c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5a2c34640 1 -- 172.21.15.92:0/1005563075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff59c101930 msgr2=0x7ff59c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff59c101930 0x7ff59c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5a2c34640 1 -- 172.21.15.92:0/1005563075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c100740 msgr2=0x7ff59c069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c100740 0x7ff59c069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5a2c34640 1 -- 172.21.15.92:0/1005563075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59c19f6f0 con 0x7ff59c1022f0 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5a1c32640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff59c101930 0x7ff59c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 0x7ff59c19ede0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff594037040 tx=0x7ff594032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5937fe640 1 -- 172.21.15.92:0/1005563075 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594041020 con 0x7ff59c1022f0 2024-03-19T01:57:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5937fe640 1 -- 172.21.15.92:0/1005563075 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff594034cd0 con 0x7ff59c1022f0 2024-03-19T01:57:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5a4697640 1 -- 172.21.15.92:0/1005563075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59c19f920 con 0x7ff59c1022f0 2024-03-19T01:57:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5937fe640 1 -- 172.21.15.92:0/1005563075 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594039040 con 0x7ff59c1022f0 2024-03-19T01:57:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.571+0000 7ff5a4697640 1 -- 172.21.15.92:0/1005563075 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59c19fe60 con 0x7ff59c1022f0 2024-03-19T01:57:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.573+0000 7ff5937fe640 1 -- 172.21.15.92:0/1005563075 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff594044020 con 0x7ff59c1022f0 2024-03-19T01:57:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.574+0000 7ff5937fe640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5880a2920 0x7ff5880a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.574+0000 7ff5a2433640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5880a2920 0x7ff5880a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.574+0000 7ff5937fe640 1 -- 172.21.15.92:0/1005563075 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff5940f0840 con 0x7ff59c1022f0 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.574+0000 7ff5a7283740 1 Processor -- start 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.574+0000 7ff5a2433640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5880a2920 0x7ff5880a4de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff59c101790 tx=0x7ff598005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a7283740 1 -- start start 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 0x55fa74301520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa743065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74306c60 0x55fa743090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a7283740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fa74265be0 con 0x55fa74301af0 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a7283740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa7420e960 con 0x55fa74306c60 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a7283740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fa74261c80 con 0x55fa74301100 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a1c32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa743065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a2433640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 0x55fa74301520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a1c32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa743065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45862/0 (socket says 172.21.15.92:45862) 2024-03-19T01:57:04.582 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a1c32640 1 -- 172.21.15.92:0/2970408374 learned_addr learned my addr 172.21.15.92:0/2970408374 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/2970408374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74306c60 0x55fa743090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a2433640 1 -- 172.21.15.92:0/2970408374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74306c60 msgr2=0x55fa743090f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a2433640 1 --2- 172.21.15.92:0/2970408374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74306c60 0x55fa743090f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a2433640 1 -- 172.21.15.92:0/2970408374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 msgr2=0x55fa743065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a2433640 1 --2- 172.21.15.92:0/2970408374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa743065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a2433640 1 -- 172.21.15.92:0/2970408374 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa74205d20 con 0x55fa74301100 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a1c32640 1 --2- 172.21.15.92:0/2970408374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa743065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.575+0000 7ff5a2433640 1 --2- 172.21.15.92:0/2970408374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 0x55fa74301520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff59800e560 tx=0x7ff598014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fb8b5f55082f44f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff590ff9640 1 -- 172.21.15.92:0/2970408374 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598004020 con 0x55fa74301100 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff590ff9640 1 -- 172.21.15.92:0/2970408374 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5980041c0 con 0x55fa74301100 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff590ff9640 1 -- 172.21.15.92:0/2970408374 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5980266c0 con 0x55fa74301100 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff5a7283740 1 -- 172.21.15.92:0/2970408374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 msgr2=0x55fa74301520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff5a7283740 1 --2- 172.21.15.92:0/2970408374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 0x55fa74301520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff59800e560 tx=0x7ff598014000 comp rx=0 tx=0).stop 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff5a7283740 1 -- 172.21.15.92:0/2970408374 shutdown_connections 2024-03-19T01:57:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff5a7283740 1 --2- 172.21.15.92:0/2970408374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74306c60 0x55fa743090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff5a7283740 1 --2- 172.21.15.92:0/2970408374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa743065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff5a7283740 1 --2- 172.21.15.92:0/2970408374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 0x55fa74301520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff5a7283740 1 -- 172.21.15.92:0/2970408374 >> 172.21.15.92:0/2970408374 conn(0x55fa742fc670 msgr2=0x55fa742fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff5a7283740 1 -- 172.21.15.92:0/2970408374 shutdown_connections 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.576+0000 7ff5a7283740 1 -- 172.21.15.92:0/2970408374 wait complete. 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a7283740 1 -- start start 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a7283740 1 Processor -- start 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 0x55fa7430b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301af0 0x55fa7430b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa743126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a7283740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fa74309c10 con 0x55fa74306c60 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a7283740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa74221db0 con 0x55fa74301100 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a7283740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fa74309d90 con 0x55fa74301af0 2024-03-19T01:57:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a7283740 10 client.0 _ll_register_callbacks cb 0x55fa74226e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a2c34640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa743126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a1c32640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301af0 0x55fa7430b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a2c34640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa743126e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45866/0 (socket says 172.21.15.92:45866) 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a2c34640 1 -- 172.21.15.92:0/3310375974 learned_addr learned my addr 172.21.15.92:0/3310375974 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a2433640 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 0x55fa7430b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a2c34640 1 -- 172.21.15.92:0/3310375974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301af0 msgr2=0x55fa7430b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301af0 0x55fa7430b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a2c34640 1 -- 172.21.15.92:0/3310375974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 msgr2=0x55fa7430b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 0x55fa7430b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a2c34640 1 -- 172.21.15.92:0/3310375974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa74220c60 con 0x55fa74306c60 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.577+0000 7ff5a1c32640 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301af0 0x55fa7430b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa743126e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff5940bcf80 tx=0x7ff5940bce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdf4411766217af server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff585ffb640 1 -- 172.21.15.92:0/3310375974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5940c13d0 con 0x55fa74306c60 2024-03-19T01:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff585ffb640 1 -- 172.21.15.92:0/3310375974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff5940bb580 con 0x55fa74306c60 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a7283740 1 -- 172.21.15.92:0/3310375974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5980041c0 con 0x55fa74306c60 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff585ffb640 1 -- 172.21.15.92:0/3310375974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5940c5990 con 0x55fa74306c60 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a7283740 10 client.15654 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a7283740 1 -- 172.21.15.92:0/3310375974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa73f7a210 con 0x55fa74306c60 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff585ffb640 1 -- 172.21.15.92:0/3310375974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5980041c0 con 0x55fa74306c60 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff585ffb640 1 -- 172.21.15.92:0/3310375974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff5940c3040 con 0x55fa74306c60 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a7283740 10 client.15654 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a7283740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a7283740 20 client.15654 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a7283740 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fa743170e0 0x55fa74319520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a7283740 4 client.15654 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.578+0000 7ff5a7283740 1 -- 172.21.15.92:0/3310375974 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa74273120 con 0x55fa743170e0 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.579+0000 7ff5a2433640 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fa743170e0 0x55fa74319520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.579+0000 7ff5a2433640 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fa743170e0 0x55fa74319520 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5a477c10afea2f server_cookie=6688553fe70a15eb in_seq=0 out_seq=0 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.579+0000 7ff585ffb640 10 client.15654 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff585ffb640 1 -- 172.21.15.92:0/3310375974 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fa74273120 con 0x55fa743170e0 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff585ffb640 10 client.15654 handle_command_reply: tid=0 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 client.15654 shutdown 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 -- 172.21.15.92:0/3310375974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 msgr2=0x55fa743126e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa743126e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff5940bcf80 tx=0x7ff5940bce80 comp rx=0 tx=0).stop 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 -- 172.21.15.92:0/3310375974 shutdown_connections 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fa743170e0 0x55fa74319520 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa743126e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff585ffb640 0 client.15654 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301af0 0x55fa7430b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 --2- 172.21.15.92:0/3310375974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 0x55fa7430b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 -- 172.21.15.92:0/3310375974 >> 172.21.15.92:0/3310375974 conn(0x55fa742fc670 msgr2=0x55fa742fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 -- 172.21.15.92:0/3310375974 shutdown_connections 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 1 -- 172.21.15.92:0/3310375974 wait complete. 2024-03-19T01:57:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.580+0000 7ff5a7283740 20 client.15654 trim_cache size 0 max 16384 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.594+0000 7ff5a7283740 1 Processor -- start 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.594+0000 7ff5a7283740 1 -- start start 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.594+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 0x55fa74306120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.594+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa74302590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.594+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74306c60 0x55fa74302ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.594+0000 7ff5a7283740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c1a8bd0 con 0x55fa74301af0 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.594+0000 7ff5a7283740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c1aa710 con 0x55fa74301100 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.594+0000 7ff5a7283740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c1a92a0 con 0x55fa74306c60 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.594+0000 7ff5a1c32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa74302590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a1c32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa74302590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45872/0 (socket says 172.21.15.92:45872) 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a1c32640 1 -- 172.21.15.92:0/2858905860 learned_addr learned my addr 172.21.15.92:0/2858905860 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a2433640 1 --2- 172.21.15.92:0/2858905860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 0x55fa74306120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/2858905860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74306c60 0x55fa74302ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a1c32640 1 -- 172.21.15.92:0/2858905860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74306c60 msgr2=0x55fa74302ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a1c32640 1 --2- 172.21.15.92:0/2858905860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74306c60 0x55fa74302ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a1c32640 1 -- 172.21.15.92:0/2858905860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 msgr2=0x55fa74306120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a1c32640 1 --2- 172.21.15.92:0/2858905860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 0x55fa74306120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a1c32640 1 -- 172.21.15.92:0/2858905860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa74046620 con 0x55fa74301af0 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff5a1c32640 1 --2- 172.21.15.92:0/2858905860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa74302590 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0128a0 tx=0x7ff58c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1206ac9ca06ff45 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff587fff640 1 -- 172.21.15.92:0/2858905860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c022070 con 0x55fa74301af0 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff587fff640 1 -- 172.21.15.92:0/2858905860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58c00f920 con 0x55fa74301af0 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.595+0000 7ff587fff640 1 -- 172.21.15.92:0/2858905860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c003a30 con 0x55fa74301af0 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 -- 172.21.15.92:0/2858905860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 msgr2=0x55fa74302590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 --2- 172.21.15.92:0/2858905860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa74302590 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0128a0 tx=0x7ff58c005f90 comp rx=0 tx=0).stop 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 -- 172.21.15.92:0/2858905860 shutdown_connections 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 --2- 172.21.15.92:0/2858905860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74306c60 0x55fa74302ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 --2- 172.21.15.92:0/2858905860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74301af0 0x55fa74302590 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 --2- 172.21.15.92:0/2858905860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301100 0x55fa74306120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 -- 172.21.15.92:0/2858905860 >> 172.21.15.92:0/2858905860 conn(0x55fa742fc670 msgr2=0x55fa742fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 -- 172.21.15.92:0/2858905860 shutdown_connections 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 -- 172.21.15.92:0/2858905860 wait complete. 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 -- start start 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 Processor -- start 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 0x55fa74379090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301af0 0x55fa743795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa74380610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c1aa040 con 0x55fa74306c60 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c1a9970 con 0x55fa74301af0 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c1a6bb0 con 0x55fa74301100 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a7283740 10 client.0 _ll_register_callbacks cb 0x7ff5940b9520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a2c34640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa74380610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a2c34640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa74380610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45874/0 (socket says 172.21.15.92:45874) 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a2c34640 1 -- 172.21.15.92:0/437555363 learned_addr learned my addr 172.21.15.92:0/437555363 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a2433640 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 0x55fa74379090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.596+0000 7ff5a1c32640 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301af0 0x55fa743795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a2c34640 1 -- 172.21.15.92:0/437555363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 msgr2=0x55fa74379090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 0x55fa74379090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a2c34640 1 -- 172.21.15.92:0/437555363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301af0 msgr2=0x55fa743795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301af0 0x55fa743795d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a2c34640 1 -- 172.21.15.92:0/437555363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa74380c00 con 0x55fa74306c60 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a2c34640 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa74380610 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff5940ee6c0 tx=0x7ff5940bd340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5dc9a77b4d60b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff585ffb640 1 -- 172.21.15.92:0/437555363 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5940bd050 con 0x55fa74306c60 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff585ffb640 1 -- 172.21.15.92:0/437555363 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff5940dd0b0 con 0x55fa74306c60 2024-03-19T01:57:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a7283740 1 -- 172.21.15.92:0/437555363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58c00f920 con 0x55fa74306c60 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff585ffb640 1 -- 172.21.15.92:0/437555363 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5940c13d0 con 0x55fa74306c60 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a7283740 10 client.15660 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a7283740 1 -- 172.21.15.92:0/437555363 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa74381f80 con 0x55fa74306c60 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff585ffb640 1 -- 172.21.15.92:0/437555363 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58c00f920 con 0x55fa74306c60 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff585ffb640 1 -- 172.21.15.92:0/437555363 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff5940c3040 con 0x55fa74306c60 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a7283740 10 client.15660 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a7283740 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a7283740 20 client.15660 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a7283740 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fa743826b0 0x55fa74384ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a7283740 4 client.15660 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.597+0000 7ff5a7283740 1 -- 172.21.15.92:0/437555363 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa7426d470 con 0x55fa743826b0 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.598+0000 7ff5a2433640 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fa743826b0 0x55fa74384ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.598+0000 7ff5a2433640 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fa743826b0 0x55fa74384ad0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6cc73215606e96a server_cookie=b2dbd4aab1617d70 in_seq=0 out_seq=0 2024-03-19T01:57:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.598+0000 7ff585ffb640 10 client.15660 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff585ffb640 1 -- 172.21.15.92:0/437555363 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa7426d470 con 0x55fa743826b0 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff585ffb640 10 client.15660 handle_command_reply: tid=0 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 client.15660 shutdown 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 -- 172.21.15.92:0/437555363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 msgr2=0x55fa74380610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa74380610 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff5940ee6c0 tx=0x7ff5940bd340 comp rx=0 tx=0).stop 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 -- 172.21.15.92:0/437555363 shutdown_connections 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55fa743826b0 0x55fa74384ad0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fa74306c60 0x55fa74380610 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa74301af0 0x55fa743795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff585ffb640 0 client.15660 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 --2- 172.21.15.92:0/437555363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa74301100 0x55fa74379090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 -- 172.21.15.92:0/437555363 >> 172.21.15.92:0/437555363 conn(0x55fa742fc670 msgr2=0x55fa742ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 -- 172.21.15.92:0/437555363 shutdown_connections 2024-03-19T01:57:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 1 -- 172.21.15.92:0/437555363 wait complete. 2024-03-19T01:57:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.604+0000 7ff5a7283740 20 client.15660 trim_cache size 0 max 16384 2024-03-19T01:57:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:57:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:57:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:57:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:57:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 -- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5880a2920 msgr2=0x7ff5880a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5880a2920 0x7ff5880a4de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff59c101790 tx=0x7ff598005f90 comp rx=0 tx=0).stop 2024-03-19T01:57:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 -- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 msgr2=0x7ff59c19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 0x7ff59c19ede0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff594037040 tx=0x7ff594032f40 comp rx=0 tx=0).stop 2024-03-19T01:57:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 -- 172.21.15.92:0/1005563075 shutdown_connections 2024-03-19T01:57:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff59c1022f0 0x7ff59c19ede0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5880a2920 0x7ff5880a4de0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff59c101930 0x7ff59c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 --2- 172.21.15.92:0/1005563075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff59c100740 0x7ff59c069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 -- 172.21.15.92:0/1005563075 >> 172.21.15.92:0/1005563075 conn(0x7ff59c0fbcb0 msgr2=0x7ff59c104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 -- 172.21.15.92:0/1005563075 shutdown_connections 2024-03-19T01:57:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:04.605+0000 7ff5a4697640 1 -- 172.21.15.92:0/1005563075 wait complete. 2024-03-19T01:57:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:04 smithi159 ceph-mon[67110]: pgmap v479: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:04 smithi092 ceph-mon[62574]: pgmap v479: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:04 smithi144 ceph-mon[67199]: pgmap v479: 97 pgs: 97 active+clean; 1.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:06 smithi159 ceph-mon[67110]: pgmap v480: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-03-19T01:57:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:06 smithi092 ceph-mon[62574]: pgmap v480: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-03-19T01:57:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:06 smithi144 ceph-mon[67199]: pgmap v480: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-03-19T01:57:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:08 smithi159 ceph-mon[67110]: pgmap v481: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-03-19T01:57:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:08 smithi092 ceph-mon[62574]: pgmap v481: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-03-19T01:57:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:08 smithi144 ceph-mon[67199]: pgmap v481: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2024-03-19T01:57:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:10 smithi159 ceph-mon[67110]: pgmap v482: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 19 op/s 2024-03-19T01:57:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:10 smithi092 ceph-mon[62574]: pgmap v482: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 19 op/s 2024-03-19T01:57:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:10 smithi144 ceph-mon[67199]: pgmap v482: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 19 op/s 2024-03-19T01:57:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:12 smithi159 ceph-mon[67110]: pgmap v483: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:57:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:12 smithi092 ceph-mon[62574]: pgmap v483: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:57:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:12 smithi144 ceph-mon[67199]: pgmap v483: 97 pgs: 97 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:57:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:14 smithi159 ceph-mon[67110]: pgmap v484: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:14 smithi092 ceph-mon[62574]: pgmap v484: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:14 smithi144 ceph-mon[67199]: pgmap v484: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:15.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:15 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:57:16.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:15 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:57:16.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:15 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:57:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:16 smithi159 ceph-mon[67110]: pgmap v485: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:57:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:57:16.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:57:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:16 smithi092 ceph-mon[62574]: pgmap v485: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:57:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:57:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:57:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:16 smithi144 ceph-mon[67199]: pgmap v485: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:17.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:57:17.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:57:17.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:57:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.691+0000 7f1e4c359640 1 -- 172.21.15.92:0/4059695686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44107520 msgr2=0x7f1e44109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 --2- 172.21.15.92:0/4059695686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44107520 0x7f1e44109970 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009ca0 tx=0x7f1e3c02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 -- 172.21.15.92:0/4059695686 shutdown_connections 2024-03-19T01:57:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 --2- 172.21.15.92:0/4059695686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44107520 0x7f1e44109970 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 --2- 172.21.15.92:0/4059695686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e44104bf0 0x7f1e44106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 --2- 172.21.15.92:0/4059695686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e441022c0 0x7f1e441046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 -- 172.21.15.92:0/4059695686 >> 172.21.15.92:0/4059695686 conn(0x7f1e440fbcb0 msgr2=0x7f1e440fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 -- 172.21.15.92:0/4059695686 shutdown_connections 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 -- 172.21.15.92:0/4059695686 wait complete. 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 Processor -- start 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 -- start start 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e441022c0 0x7f1e44197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44104bf0 0x7f1e44198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e44107520 0x7f1e4419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4410d4e0 con 0x7f1e44104bf0 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4410d360 con 0x7f1e44107520 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.692+0000 7f1e4c359640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4410d660 con 0x7f1e441022c0 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a0f5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e441022c0 0x7f1e44197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e498f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44104bf0 0x7f1e44198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a0f5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e441022c0 0x7f1e44197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:39122/0 (socket says 172.21.15.92:39122) 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e498f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44104bf0 0x7f1e44198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46800/0 (socket says 172.21.15.92:46800) 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/1598009783 learned_addr learned my addr 172.21.15.92:0/1598009783 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a8f6640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e44107520 0x7f1e4419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/1598009783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e44107520 msgr2=0x7f1e4419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e44107520 0x7f1e4419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44104bf0 msgr2=0x7f1e44198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44104bf0 0x7f1e44198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/1598009783 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4419f6e0 con 0x7f1e441022c0 2024-03-19T01:57:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e498f4640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44104bf0 0x7f1e44198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e441022c0 0x7f1e44197ac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000e9f0 tx=0x7f1e4000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e3b7fe640 1 -- 172.21.15.92:0/1598009783 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4000cd80 con 0x7f1e441022c0 2024-03-19T01:57:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e3b7fe640 1 -- 172.21.15.92:0/1598009783 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e40005500 con 0x7f1e441022c0 2024-03-19T01:57:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e3b7fe640 1 -- 172.21.15.92:0/1598009783 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40010690 con 0x7f1e441022c0 2024-03-19T01:57:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4c359640 1 -- 172.21.15.92:0/1598009783 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4419f9d0 con 0x7f1e441022c0 2024-03-19T01:57:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.693+0000 7f1e4c359640 1 -- 172.21.15.92:0/1598009783 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4419ff20 con 0x7f1e441022c0 2024-03-19T01:57:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.695+0000 7f1e3b7fe640 1 -- 172.21.15.92:0/1598009783 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f1e400046c0 con 0x7f1e441022c0 2024-03-19T01:57:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.696+0000 7f1e3b7fe640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1e300a2920 0x7f1e300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.696+0000 7f1e498f4640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1e300a2920 0x7f1e300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.696+0000 7f1e3b7fe640 1 -- 172.21.15.92:0/1598009783 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1e400cedd0 con 0x7f1e441022c0 2024-03-19T01:57:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.696+0000 7f1e498f4640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1e300a2920 0x7f1e300a4de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1e44198fd0 tx=0x7f1e34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4f016740 1 Processor -- start 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4f016740 1 -- start start 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582100 0x55bf22582520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf225875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf2258a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4f016740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf224e6be0 con 0x55bf22587c60 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4f016740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf2248f960 con 0x55bf22582af0 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4f016740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf224e2c80 con 0x55bf22582100 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4a8f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf2258a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4a8f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf2258a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46802/0 (socket says 172.21.15.92:46802) 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4a8f6640 1 -- 172.21.15.92:0/3111674676 learned_addr learned my addr 172.21.15.92:0/3111674676 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e498f4640 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf225875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.697+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582100 0x55bf22582520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e4a8f6640 1 -- 172.21.15.92:0/3111674676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582100 msgr2=0x55bf22582520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e4a8f6640 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582100 0x55bf22582520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e4a8f6640 1 -- 172.21.15.92:0/3111674676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 msgr2=0x55bf225875f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e4a8f6640 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf225875f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e4a8f6640 1 -- 172.21.15.92:0/3111674676 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf22486d20 con 0x55bf22587c60 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e498f4640 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf225875f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e4a8f6640 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf2258a0f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009ca0 tx=0x7f1e3c0323c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0b07672bfd8a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e38bfe640 1 -- 172.21.15.92:0/3111674676 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c043070 con 0x55bf22587c60 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e38bfe640 1 -- 172.21.15.92:0/3111674676 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c032c90 con 0x55bf22587c60 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.698+0000 7f1e38bfe640 1 -- 172.21.15.92:0/3111674676 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c039040 con 0x55bf22587c60 2024-03-19T01:57:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 -- 172.21.15.92:0/3111674676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 msgr2=0x55bf2258a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf2258a0f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009ca0 tx=0x7f1e3c0323c0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 -- 172.21.15.92:0/3111674676 shutdown_connections 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf2258a0f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf225875f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3111674676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582100 0x55bf22582520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 -- 172.21.15.92:0/3111674676 >> 172.21.15.92:0/3111674676 conn(0x55bf2257d670 msgr2=0x55bf2257da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 -- 172.21.15.92:0/3111674676 shutdown_connections 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 -- 172.21.15.92:0/3111674676 wait complete. 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 -- start start 2024-03-19T01:57:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 Processor -- start 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf2258c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 0x55bf225935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf2258abb0 con 0x55bf22582100 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf224a2db0 con 0x55bf22582af0 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf2258ad30 con 0x55bf22587c60 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4f016740 10 client.0 _ll_register_callbacks cb 0x55bf224a7e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4a0f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4a0f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46804/0 (socket says 172.21.15.92:46804) 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.699+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/3942497312 learned_addr learned my addr 172.21.15.92:0/3942497312 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e498f4640 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf2258c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4a8f6640 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 0x55bf225935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/3942497312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 msgr2=0x55bf225935a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 0x55bf225935a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/3942497312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 msgr2=0x55bf2258c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf2258c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/3942497312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf224a1c60 con 0x55bf22582100 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e498f4640 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf2258c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c020 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1e400ccc50 tx=0x7f1e4009b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878b8d7d97c0755b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e2dffb640 1 -- 172.21.15.92:0/3942497312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b3020 con 0x55bf22582100 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e2dffb640 1 -- 172.21.15.92:0/3942497312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1e400a3a90 con 0x55bf22582100 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4f016740 1 -- 172.21.15.92:0/3942497312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c032c90 con 0x55bf22582100 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e2dffb640 1 -- 172.21.15.92:0/3942497312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e400a5070 con 0x55bf22582100 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4f016740 10 client.15672 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e4f016740 1 -- 172.21.15.92:0/3942497312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf221fb210 con 0x55bf22582100 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.700+0000 7f1e2dffb640 1 -- 172.21.15.92:0/3942497312 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c032c90 con 0x55bf22582100 2024-03-19T01:57:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e2dffb640 1 -- 172.21.15.92:0/3942497312 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1e40097040 con 0x55bf22582100 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e4f016740 10 client.15672 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e4f016740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e4f016740 20 client.15672 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf22597ef0 0x55bf2259a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e4f016740 4 client.15672 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e4f016740 1 -- 172.21.15.92:0/3942497312 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf224f4120 con 0x55bf22597ef0 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e498f4640 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf22597ef0 0x55bf2259a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e498f4640 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf22597ef0 0x55bf2259a3b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b721641305df6f87 server_cookie=a75b7fa8b2c8d910 in_seq=0 out_seq=0 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.701+0000 7f1e2dffb640 10 client.15672 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e2dffb640 1 -- 172.21.15.92:0/3942497312 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bf224f4120 con 0x55bf22597ef0 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e2dffb640 10 client.15672 handle_command_reply: tid=0 2024-03-19T01:57:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 client.15672 shutdown 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 -- 172.21.15.92:0/3942497312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 msgr2=0x55bf2258c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c020 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1e400ccc50 tx=0x7f1e4009b0e0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 -- 172.21.15.92:0/3942497312 shutdown_connections 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf22597ef0 0x55bf2259a3b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 0x55bf225935a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf2258c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e2dffb640 0 client.15672 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 --2- 172.21.15.92:0/3942497312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c020 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 -- 172.21.15.92:0/3942497312 >> 172.21.15.92:0/3942497312 conn(0x55bf2257d670 msgr2=0x55bf2257e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 -- 172.21.15.92:0/3942497312 shutdown_connections 2024-03-19T01:57:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 1 -- 172.21.15.92:0/3942497312 wait complete. 2024-03-19T01:57:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.702+0000 7f1e4f016740 20 client.15672 trim_cache size 0 max 16384 2024-03-19T01:57:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.716+0000 7f1e4f016740 1 Processor -- start 2024-03-19T01:57:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4f016740 1 -- start start 2024-03-19T01:57:17.725 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf22593560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 0x55bf22583590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4f016740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441a8c30 con 0x55bf22582100 2024-03-19T01:57:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4f016740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441aa770 con 0x55bf22582af0 2024-03-19T01:57:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4f016740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441a9300 con 0x55bf22587c60 2024-03-19T01:57:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a0f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a0f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46816/0 (socket says 172.21.15.92:46816) 2024-03-19T01:57:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/2484649519 learned_addr learned my addr 172.21.15.92:0/2484649519 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e498f4640 1 --2- 172.21.15.92:0/2484649519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf22593560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a8f6640 1 --2- 172.21.15.92:0/2484649519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 0x55bf22583590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/2484649519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 msgr2=0x55bf22583590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/2484649519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 0x55bf22583590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/2484649519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 msgr2=0x55bf22593560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/2484649519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf22593560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a0f5640 1 -- 172.21.15.92:0/2484649519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf222c7620 con 0x55bf22582100 2024-03-19T01:57:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/2484649519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c340 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1e400ba1d0 tx=0x7f1e40005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad93c4e0ca90e64 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e2ffff640 1 -- 172.21.15.92:0/2484649519 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400af070 con 0x55bf22582100 2024-03-19T01:57:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e2ffff640 1 -- 172.21.15.92:0/2484649519 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e400ba020 con 0x55bf22582100 2024-03-19T01:57:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.717+0000 7f1e2ffff640 1 -- 172.21.15.92:0/2484649519 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b3030 con 0x55bf22582100 2024-03-19T01:57:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 -- 172.21.15.92:0/2484649519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 msgr2=0x55bf2258c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2484649519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c340 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1e400ba1d0 tx=0x7f1e40005760 comp rx=0 tx=0).stop 2024-03-19T01:57:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 -- 172.21.15.92:0/2484649519 shutdown_connections 2024-03-19T01:57:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2484649519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22587c60 0x55bf22583590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2484649519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582af0 0x55bf22593560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2484649519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22582100 0x55bf2258c340 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 -- 172.21.15.92:0/2484649519 >> 172.21.15.92:0/2484649519 conn(0x55bf2257d670 msgr2=0x55bf2258bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 -- 172.21.15.92:0/2484649519 shutdown_connections 2024-03-19T01:57:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 -- 172.21.15.92:0/2484649519 wait complete. 2024-03-19T01:57:17.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 -- start start 2024-03-19T01:57:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 Processor -- start 2024-03-19T01:57:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582100 0x55bf22592e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582af0 0x55bf225933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf225914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441aa0a0 con 0x55bf22587c60 2024-03-19T01:57:17.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441a99d0 con 0x55bf22582100 2024-03-19T01:57:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441a6c10 con 0x55bf22582af0 2024-03-19T01:57:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.718+0000 7f1e4f016740 10 client.0 _ll_register_callbacks cb 0x7f1e40097a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a8f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf225914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a8f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf225914d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46828/0 (socket says 172.21.15.92:46828) 2024-03-19T01:57:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a8f6640 1 -- 172.21.15.92:0/2220650122 learned_addr learned my addr 172.21.15.92:0/2220650122 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582100 0x55bf22592e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e498f4640 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582af0 0x55bf225933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a8f6640 1 -- 172.21.15.92:0/2220650122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582af0 msgr2=0x55bf225933b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a8f6640 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582af0 0x55bf225933b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a8f6640 1 -- 172.21.15.92:0/2220650122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582100 msgr2=0x55bf22592e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a8f6640 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582100 0x55bf22592e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a8f6640 1 -- 172.21.15.92:0/2220650122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf22591c40 con 0x55bf22587c60 2024-03-19T01:57:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582100 0x55bf22592e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4a8f6640 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf225914d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005e30 tx=0x7f1e3c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5b0750d3110ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e2dffb640 1 -- 172.21.15.92:0/2220650122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c043040 con 0x55bf22587c60 2024-03-19T01:57:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e2dffb640 1 -- 172.21.15.92:0/2220650122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1e3c032890 con 0x55bf22587c60 2024-03-19T01:57:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4f016740 1 -- 172.21.15.92:0/2220650122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e400ba020 con 0x55bf22587c60 2024-03-19T01:57:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e2dffb640 1 -- 172.21.15.92:0/2220650122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e3c039040 con 0x55bf22587c60 2024-03-19T01:57:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4f016740 10 client.15678 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e4f016740 1 -- 172.21.15.92:0/2220650122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf225fedd0 con 0x55bf22587c60 2024-03-19T01:57:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.719+0000 7f1e2dffb640 1 -- 172.21.15.92:0/2220650122 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e400ba020 con 0x55bf22587c60 2024-03-19T01:57:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e2dffb640 1 -- 172.21.15.92:0/2220650122 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1e3c0044f0 con 0x55bf22587c60 2024-03-19T01:57:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e4f016740 10 client.15678 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e4f016740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e4f016740 20 client.15678 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf225ff3f0 0x55bf226017e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e4f016740 4 client.15678 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:57:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e4f016740 1 -- 172.21.15.92:0/2220650122 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf224ee470 con 0x55bf225ff3f0 2024-03-19T01:57:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf225ff3f0 0x55bf226017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e4a0f5640 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf225ff3f0 0x55bf226017e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65be9df5f42136a server_cookie=891b65c5a701d3c5 in_seq=0 out_seq=0 2024-03-19T01:57:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.720+0000 7f1e2dffb640 10 client.15678 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e2dffb640 1 -- 172.21.15.92:0/2220650122 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf224ee470 con 0x55bf225ff3f0 2024-03-19T01:57:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e2dffb640 10 client.15678 handle_command_reply: tid=0 2024-03-19T01:57:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 client.15678 shutdown 2024-03-19T01:57:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 -- 172.21.15.92:0/2220650122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 msgr2=0x55bf225914d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf225914d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005e30 tx=0x7f1e3c005bd0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 -- 172.21.15.92:0/2220650122 shutdown_connections 2024-03-19T01:57:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf225ff3f0 0x55bf226017e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf22587c60 0x55bf225914d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf22582af0 0x55bf225933b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e2dffb640 0 client.15678 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 --2- 172.21.15.92:0/2220650122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf22582100 0x55bf22592e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 -- 172.21.15.92:0/2220650122 >> 172.21.15.92:0/2220650122 conn(0x55bf2257d670 msgr2=0x55bf22583590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 -- 172.21.15.92:0/2220650122 shutdown_connections 2024-03-19T01:57:17.740 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 1 -- 172.21.15.92:0/2220650122 wait complete. 2024-03-19T01:57:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.727+0000 7f1e4f016740 20 client.15678 trim_cache size 0 max 16384 2024-03-19T01:57:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:57:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:57:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:57:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:57:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 -- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1e300a2920 msgr2=0x7f1e300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1e300a2920 0x7f1e300a4de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1e44198fd0 tx=0x7f1e34005f90 comp rx=0 tx=0).stop 2024-03-19T01:57:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 -- 172.21.15.92:0/1598009783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e441022c0 msgr2=0x7f1e44197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e441022c0 0x7f1e44197ac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000e9f0 tx=0x7f1e4000edf0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 -- 172.21.15.92:0/1598009783 shutdown_connections 2024-03-19T01:57:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e44107520 0x7f1e4419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e44104bf0 0x7f1e44198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1e300a2920 0x7f1e300a4de0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 --2- 172.21.15.92:0/1598009783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e441022c0 0x7f1e44197ac0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 -- 172.21.15.92:0/1598009783 >> 172.21.15.92:0/1598009783 conn(0x7f1e440fbcb0 msgr2=0x7f1e44102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 -- 172.21.15.92:0/1598009783 shutdown_connections 2024-03-19T01:57:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:17.728+0000 7f1e4c359640 1 -- 172.21.15.92:0/1598009783 wait complete. 2024-03-19T01:57:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:18 smithi159 ceph-mon[67110]: pgmap v486: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:57:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:18 smithi092 ceph-mon[62574]: pgmap v486: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:57:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:18 smithi144 ceph-mon[67199]: pgmap v486: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:57:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:20 smithi159 ceph-mon[67110]: pgmap v487: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:57:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:20 smithi092 ceph-mon[62574]: pgmap v487: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:57:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:20 smithi144 ceph-mon[67199]: pgmap v487: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:57:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:22 smithi159 ceph-mon[67110]: pgmap v488: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T01:57:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:22 smithi092 ceph-mon[62574]: pgmap v488: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T01:57:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:22 smithi144 ceph-mon[67199]: pgmap v488: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T01:57:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:24 smithi159 ceph-mon[67110]: pgmap v489: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:57:25.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:24 smithi092 ceph-mon[62574]: pgmap v489: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:57:25.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:24 smithi144 ceph-mon[67199]: pgmap v489: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:57:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:26 smithi159 ceph-mon[67110]: pgmap v490: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:57:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:26 smithi092 ceph-mon[62574]: pgmap v490: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:57:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:26 smithi144 ceph-mon[67199]: pgmap v490: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:57:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:28 smithi159 ceph-mon[67110]: pgmap v491: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:57:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:28 smithi092 ceph-mon[62574]: pgmap v491: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:57:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:28 smithi144 ceph-mon[67199]: pgmap v491: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.814+0000 7fc679fab640 1 -- 172.21.15.92:0/4276974217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674101880 msgr2=0x7fc674101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.814+0000 7fc679fab640 1 --2- 172.21.15.92:0/4276974217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674101880 0x7fc674101ca0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc664009900 tx=0x7fc66402bc30 comp rx=0 tx=0).stop 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.814+0000 7fc679fab640 1 -- 172.21.15.92:0/4276974217 shutdown_connections 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.814+0000 7fc679fab640 1 --2- 172.21.15.92:0/4276974217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc674103430 0x7fc674109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.814+0000 7fc679fab640 1 --2- 172.21.15.92:0/4276974217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc674102a70 0x7fc674102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.814+0000 7fc679fab640 1 --2- 172.21.15.92:0/4276974217 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674101880 0x7fc674101ca0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.814+0000 7fc679fab640 1 -- 172.21.15.92:0/4276974217 >> 172.21.15.92:0/4276974217 conn(0x7fc6740fcdf0 msgr2=0x7fc6740ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.814+0000 7fc679fab640 1 -- 172.21.15.92:0/4276974217 shutdown_connections 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.814+0000 7fc679fab640 1 -- 172.21.15.92:0/4276974217 wait complete. 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc679fab640 1 Processor -- start 2024-03-19T01:57:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc679fab640 1 -- start start 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc679fab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc674101880 0x7fc674197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc679fab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674102a70 0x7fc674197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc679fab640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc674103430 0x7fc67419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc679fab640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc67410d4d0 con 0x7fc674102a70 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc679fab640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc67410d350 con 0x7fc674101880 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc679fab640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc67410d650 con 0x7fc674103430 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc672ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674102a70 0x7fc674197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc672ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674102a70 0x7fc674197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51918/0 (socket says 172.21.15.92:51918) 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc672ffd640 1 -- 172.21.15.92:0/357448240 learned_addr learned my addr 172.21.15.92:0/357448240 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc673fff640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc674103430 0x7fc67419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.815+0000 7fc6737fe640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc674101880 0x7fc674197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc672ffd640 1 -- 172.21.15.92:0/357448240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc674103430 msgr2=0x7fc67419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc672ffd640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc674103430 0x7fc67419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc672ffd640 1 -- 172.21.15.92:0/357448240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc674101880 msgr2=0x7fc674197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc672ffd640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc674101880 0x7fc674197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc672ffd640 1 -- 172.21.15.92:0/357448240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67419f6b0 con 0x7fc674102a70 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc6737fe640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc674101880 0x7fc674197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc672ffd640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674102a70 0x7fc674197fd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00e9f0 tx=0x7fc65c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc670ff9640 1 -- 172.21.15.92:0/357448240 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c00cd80 con 0x7fc674102a70 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc679fab640 1 -- 172.21.15.92:0/357448240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67419f9a0 con 0x7fc674102a70 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc670ff9640 1 -- 172.21.15.92:0/357448240 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc65c010910 con 0x7fc674102a70 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc670ff9640 1 -- 172.21.15.92:0/357448240 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c0188f0 con 0x7fc674102a70 2024-03-19T01:57:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.816+0000 7fc679fab640 1 -- 172.21.15.92:0/357448240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67419fef0 con 0x7fc674102a70 2024-03-19T01:57:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc670ff9640 1 -- 172.21.15.92:0/357448240 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fc65c010ac0 con 0x7fc674102a70 2024-03-19T01:57:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc67cb8a740 1 Processor -- start 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc67cb8a740 1 -- start start 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5510 0x5650e10f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5f00 0x5650e10faa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 0x5650e10fd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5650e1055be0 con 0x5650e10f5f00 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650e0ffe960 con 0x5650e10f5510 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650e1051c80 con 0x5650e10fb070 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5f00 0x5650e10faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5f00 0x5650e10faa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51930/0 (socket says 172.21.15.92:51930) 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc6737fe640 1 -- 172.21.15.92:0/3348673529 learned_addr learned my addr 172.21.15.92:0/3348673529 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc673fff640 1 --2- 172.21.15.92:0/3348673529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5510 0x5650e10f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc672ffd640 1 --2- 172.21.15.92:0/3348673529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 0x5650e10fd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc6737fe640 1 -- 172.21.15.92:0/3348673529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 msgr2=0x5650e10fd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc6737fe640 1 --2- 172.21.15.92:0/3348673529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 0x5650e10fd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc6737fe640 1 -- 172.21.15.92:0/3348673529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5510 msgr2=0x5650e10f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc670ff9640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc6600a28a0 0x7fc6600a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc6737fe640 1 --2- 172.21.15.92:0/3348673529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5510 0x5650e10f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc6737fe640 1 -- 172.21.15.92:0/3348673529 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e0ff5d20 con 0x5650e10f5f00 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc673fff640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc6600a28a0 0x7fc6600a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc6737fe640 1 --2- 172.21.15.92:0/3348673529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5f00 0x5650e10faa00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc664009900 tx=0x7fc66402cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20f7c8ee53af8c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.819+0000 7fc670ff9640 1 -- 172.21.15.92:0/357448240 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc65c0cee00 con 0x7fc674102a70 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.820+0000 7fc669ffb640 1 -- 172.21.15.92:0/3348673529 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664005d50 con 0x5650e10f5f00 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.820+0000 7fc669ffb640 1 -- 172.21.15.92:0/3348673529 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc664005520 con 0x5650e10f5f00 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.820+0000 7fc669ffb640 1 -- 172.21.15.92:0/3348673529 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664004c90 con 0x5650e10f5f00 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.820+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3348673529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5f00 msgr2=0x5650e10faa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.820+0000 7fc673fff640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc6600a28a0 0x7fc6600a4d60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009fd0 tx=0x7fc66c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.820+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3348673529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5f00 0x5650e10faa00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc664009900 tx=0x7fc66402cd30 comp rx=0 tx=0).stop 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3348673529 shutdown_connections 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3348673529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 0x5650e10fd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3348673529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5f00 0x5650e10faa00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3348673529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5510 0x5650e10f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3348673529 >> 172.21.15.92:0/3348673529 conn(0x5650e10ec670 msgr2=0x5650e10eca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3348673529 shutdown_connections 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3348673529 wait complete. 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 -- start start 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 Processor -- start 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5510 0x5650e10ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 0x5650e10ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10fb070 0x5650e1106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5650e10fdfc0 con 0x5650e10fb070 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650e1011db0 con 0x5650e10f5f00 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650e10fe140 con 0x5650e10f5510 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc67cb8a740 10 client.0 _ll_register_callbacks cb 0x5650e1016e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc673fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10fb070 0x5650e1106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5510 0x5650e10ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc673fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10fb070 0x5650e1106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51932/0 (socket says 172.21.15.92:51932) 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.821+0000 7fc673fff640 1 -- 172.21.15.92:0/2732441050 learned_addr learned my addr 172.21.15.92:0/2732441050 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc672ffd640 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 0x5650e10ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc673fff640 1 -- 172.21.15.92:0/2732441050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5510 msgr2=0x5650e10ff4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc673fff640 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5510 0x5650e10ff4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc673fff640 1 -- 172.21.15.92:0/2732441050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 msgr2=0x5650e10ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc673fff640 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 0x5650e10ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc673fff640 1 -- 172.21.15.92:0/2732441050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e1010c60 con 0x5650e10fb070 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc6737fe640 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5510 0x5650e10ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc673fff640 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10fb070 0x5650e1106a30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00fb90 tx=0x7fc66c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fa8543649f77e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc652ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc652ffd640 1 -- 172.21.15.92:0/2732441050 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c01eac0 con 0x5650e10fb070 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc652ffd640 1 -- 172.21.15.92:0/2732441050 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc66c01ec60 con 0x5650e10fb070 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc67cb8a740 1 -- 172.21.15.92:0/2732441050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc664005520 con 0x5650e10fb070 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc652ffd640 1 -- 172.21.15.92:0/2732441050 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc66c01ee60 con 0x5650e10fb070 2024-03-19T01:57:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc67cb8a740 10 client.15690 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc67cb8a740 1 -- 172.21.15.92:0/2732441050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650e0d6a210 con 0x5650e10fb070 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.822+0000 7fc652ffd640 1 -- 172.21.15.92:0/2732441050 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc664005520 con 0x5650e10fb070 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc652ffd640 1 -- 172.21.15.92:0/2732441050 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc66c0284f0 con 0x5650e10fb070 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc67cb8a740 10 client.15690 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc67cb8a740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc67cb8a740 20 client.15690 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5650e110b380 0x5650e110d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc67cb8a740 4 client.15690 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc67cb8a740 1 -- 172.21.15.92:0/2732441050 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650e1063120 con 0x5650e110b380 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc6737fe640 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5650e110b380 0x5650e110d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc652ffd640 10 client.15690 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.823+0000 7fc6737fe640 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5650e110b380 0x5650e110d7b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aab22b9e83de6d5 server_cookie=1e312e31e4db5113 in_seq=0 out_seq=0 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc652ffd640 1 -- 172.21.15.92:0/2732441050 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5650e1063120 con 0x5650e110b380 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc652ffd640 10 client.15690 handle_command_reply: tid=0 2024-03-19T01:57:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 client.15690 shutdown 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 -- 172.21.15.92:0/2732441050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10fb070 msgr2=0x5650e1106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10fb070 0x5650e1106a30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00fb90 tx=0x7fc66c004ab0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 -- 172.21.15.92:0/2732441050 shutdown_connections 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5650e110b380 0x5650e110d7b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10fb070 0x5650e1106a30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc652ffd640 0 client.15690 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 0x5650e10ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/2732441050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5510 0x5650e10ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 -- 172.21.15.92:0/2732441050 >> 172.21.15.92:0/2732441050 conn(0x5650e10ec670 msgr2=0x5650e10ed8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 -- 172.21.15.92:0/2732441050 shutdown_connections 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 1 -- 172.21.15.92:0/2732441050 wait complete. 2024-03-19T01:57:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.824+0000 7fc67cb8a740 20 client.15690 trim_cache size 0 max 16384 2024-03-19T01:57:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc67cb8a740 1 Processor -- start 2024-03-19T01:57:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc67cb8a740 1 -- start start 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e10f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 0x5650e11069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 0x5650e10f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6741a6a60 con 0x5650e10f5510 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6741a85a0 con 0x5650e10f5f00 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6741a7130 con 0x5650e10fb070 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e10f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e10f69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51940/0 (socket says 172.21.15.92:51940) 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc6737fe640 1 -- 172.21.15.92:0/4025457552 learned_addr learned my addr 172.21.15.92:0/4025457552 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc672ffd640 1 --2- 172.21.15.92:0/4025457552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 0x5650e11069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc673fff640 1 --2- 172.21.15.92:0/4025457552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 0x5650e10f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc6737fe640 1 -- 172.21.15.92:0/4025457552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 msgr2=0x5650e10f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc6737fe640 1 --2- 172.21.15.92:0/4025457552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 0x5650e10f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc6737fe640 1 -- 172.21.15.92:0/4025457552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 msgr2=0x5650e11069f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc6737fe640 1 --2- 172.21.15.92:0/4025457552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 0x5650e11069f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc6737fe640 1 -- 172.21.15.92:0/4025457552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e0e36620 con 0x5650e10f5510 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.839+0000 7fc6737fe640 1 --2- 172.21.15.92:0/4025457552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e10f69a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc66402c7b0 tx=0x7fc66402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df00189e02a36ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc6697fa640 1 -- 172.21.15.92:0/4025457552 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664009520 con 0x5650e10f5510 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc6697fa640 1 -- 172.21.15.92:0/4025457552 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc664005520 con 0x5650e10f5510 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc6697fa640 1 -- 172.21.15.92:0/4025457552 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6640046c0 con 0x5650e10f5510 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 -- 172.21.15.92:0/4025457552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 msgr2=0x5650e10f69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/4025457552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e10f69a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc66402c7b0 tx=0x7fc66402bc30 comp rx=0 tx=0).stop 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 -- 172.21.15.92:0/4025457552 shutdown_connections 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/4025457552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10fb070 0x5650e10f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/4025457552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10f5f00 0x5650e11069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/4025457552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e10f69a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 -- 172.21.15.92:0/4025457552 >> 172.21.15.92:0/4025457552 conn(0x5650e10ec670 msgr2=0x5650e10eb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 -- 172.21.15.92:0/4025457552 shutdown_connections 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 -- 172.21.15.92:0/4025457552 wait complete. 2024-03-19T01:57:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.840+0000 7fc67cb8a740 1 -- start start 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc67cb8a740 1 Processor -- start 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e1104560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5f00 0x5650e1104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc67cb8a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10fb070 0x5650e1105a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6741a7ed0 con 0x5650e10f5510 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6741a7800 con 0x5650e10fb070 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc67cb8a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6741a4a40 con 0x5650e10f5f00 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc67cb8a740 10 client.0 _ll_register_callbacks cb 0x7fc66c009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e1104560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e1104560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51946/0 (socket says 172.21.15.92:51946) 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc6737fe640 1 -- 172.21.15.92:0/3208610843 learned_addr learned my addr 172.21.15.92:0/3208610843 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc672ffd640 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5f00 0x5650e1104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc673fff640 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10fb070 0x5650e1105a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc6737fe640 1 -- 172.21.15.92:0/3208610843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5f00 msgr2=0x5650e1104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc6737fe640 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5f00 0x5650e1104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc6737fe640 1 -- 172.21.15.92:0/3208610843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10fb070 msgr2=0x5650e1105a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc6737fe640 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10fb070 0x5650e1105a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.841+0000 7fc6737fe640 1 -- 172.21.15.92:0/3208610843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e1106050 con 0x5650e10f5510 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc6737fe640 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e1104560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc664000c00 tx=0x7fc664005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b403e48407b786 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc652ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc652ffd640 1 -- 172.21.15.92:0/3208610843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664056020 con 0x5650e10f5510 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc652ffd640 1 -- 172.21.15.92:0/3208610843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc6640520c0 con 0x5650e10f5510 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3208610843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc664005520 con 0x5650e10f5510 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc652ffd640 1 -- 172.21.15.92:0/3208610843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6640522c0 con 0x5650e10f5510 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc67cb8a740 10 client.15696 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3208610843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650e1171a30 con 0x5650e10f5510 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc652ffd640 1 -- 172.21.15.92:0/3208610843 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc664005520 con 0x5650e10f5510 2024-03-19T01:57:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc652ffd640 1 -- 172.21.15.92:0/3208610843 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc6640035f0 con 0x5650e10f5510 2024-03-19T01:57:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc67cb8a740 10 client.15696 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc67cb8a740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc67cb8a740 20 client.15696 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5650e1171f00 0x5650e1174320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc67cb8a740 4 client.15696 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:57:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3208610843 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650e10ecff0 con 0x5650e1171f00 2024-03-19T01:57:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.842+0000 7fc672ffd640 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5650e1171f00 0x5650e1174320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.843+0000 7fc652ffd640 10 client.15696 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.843+0000 7fc672ffd640 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5650e1171f00 0x5650e1174320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a972624e84e5d3 server_cookie=afa3cc849703ad27 in_seq=0 out_seq=0 2024-03-19T01:57:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc652ffd640 1 -- 172.21.15.92:0/3208610843 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650e10ecff0 con 0x5650e1171f00 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc652ffd640 10 client.15696 handle_command_reply: tid=0 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 client.15696 shutdown 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3208610843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 msgr2=0x5650e1104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e1104560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc664000c00 tx=0x7fc664005760 comp rx=0 tx=0).stop 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3208610843 shutdown_connections 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5650e1171f00 0x5650e1174320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650e10fb070 0x5650e1105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e10f5f00 0x5650e1104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 --2- 172.21.15.92:0/3208610843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5650e10f5510 0x5650e1104560 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc652ffd640 0 client.15696 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3208610843 >> 172.21.15.92:0/3208610843 conn(0x5650e10ec670 msgr2=0x5650e10ef8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:30.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3208610843 shutdown_connections 2024-03-19T01:57:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 1 -- 172.21.15.92:0/3208610843 wait complete. 2024-03-19T01:57:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.849+0000 7fc67cb8a740 20 client.15696 trim_cache size 0 max 16384 2024-03-19T01:57:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:57:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:57:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:57:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:57:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 -- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc6600a28a0 msgr2=0x7fc6600a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc6600a28a0 0x7fc6600a4d60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009fd0 tx=0x7fc66c004a40 comp rx=0 tx=0).stop 2024-03-19T01:57:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 -- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674102a70 msgr2=0x7fc674197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674102a70 0x7fc674197fd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00e9f0 tx=0x7fc65c00eeb0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 -- 172.21.15.92:0/357448240 shutdown_connections 2024-03-19T01:57:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fc6600a28a0 0x7fc6600a4d60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc674103430 0x7fc67419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc674102a70 0x7fc674197fd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 --2- 172.21.15.92:0/357448240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc674101880 0x7fc674197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 -- 172.21.15.92:0/357448240 >> 172.21.15.92:0/357448240 conn(0x7fc6740fcdf0 msgr2=0x7fc674105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 -- 172.21.15.92:0/357448240 shutdown_connections 2024-03-19T01:57:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:30.850+0000 7fc679fab640 1 -- 172.21.15.92:0/357448240 wait complete. 2024-03-19T01:57:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:30 smithi159 ceph-mon[67110]: pgmap v492: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:30 smithi092 ceph-mon[62574]: pgmap v492: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:30 smithi144 ceph-mon[67199]: pgmap v492: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:32.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:32 smithi159 ceph-mon[67110]: pgmap v493: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T01:57:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:32 smithi092 ceph-mon[62574]: pgmap v493: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T01:57:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:32 smithi144 ceph-mon[67199]: pgmap v493: 97 pgs: 97 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T01:57:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:34 smithi159 ceph-mon[67110]: pgmap v494: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:34 smithi092 ceph-mon[62574]: pgmap v494: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:34 smithi144 ceph-mon[67199]: pgmap v494: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:36 smithi159 ceph-mon[67110]: pgmap v495: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:36 smithi092 ceph-mon[62574]: pgmap v495: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:36 smithi144 ceph-mon[67199]: pgmap v495: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:38 smithi159 ceph-mon[67110]: pgmap v496: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:57:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:38 smithi092 ceph-mon[62574]: pgmap v496: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:57:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:38 smithi144 ceph-mon[67199]: pgmap v496: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:57:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:40 smithi159 ceph-mon[67110]: pgmap v497: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:57:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:40 smithi092 ceph-mon[62574]: pgmap v497: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:57:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:40 smithi144 ceph-mon[67199]: pgmap v497: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:57:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:42 smithi159 ceph-mon[67110]: pgmap v498: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:57:43.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:42 smithi092 ceph-mon[62574]: pgmap v498: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:57:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:42 smithi144 ceph-mon[67199]: pgmap v498: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T01:57:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.939+0000 7f0347552640 1 -- 172.21.15.92:0/90033450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340107580 msgr2=0x7f0340109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.939+0000 7f0347552640 1 --2- 172.21.15.92:0/90033450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340107580 0x7f0340109970 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0338009ca0 tx=0x7f033802c0f0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.939+0000 7f0347552640 1 -- 172.21.15.92:0/90033450 shutdown_connections 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.939+0000 7f0347552640 1 --2- 172.21.15.92:0/90033450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340107580 0x7f0340109970 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.939+0000 7f0347552640 1 --2- 172.21.15.92:0/90033450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340104c50 0x7f0340107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.939+0000 7f0347552640 1 --2- 172.21.15.92:0/90033450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0340102320 0x7f0340104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.939+0000 7f0347552640 1 -- 172.21.15.92:0/90033450 >> 172.21.15.92:0/90033450 conn(0x7f03400fbcb0 msgr2=0x7f03400fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.939+0000 7f0347552640 1 -- 172.21.15.92:0/90033450 shutdown_connections 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.939+0000 7f0347552640 1 -- 172.21.15.92:0/90033450 wait complete. 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0347552640 1 Processor -- start 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0347552640 1 -- start start 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0347552640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340102320 0x7f0340101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0347552640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340104c50 0x7f03401002d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0347552640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0340107580 0x7f0340100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0347552640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f034010d4e0 con 0x7f0340102320 2024-03-19T01:57:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0347552640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f034010d360 con 0x7f0340104c50 2024-03-19T01:57:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0347552640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f034010d660 con 0x7f0340107580 2024-03-19T01:57:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03452ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340102320 0x7f0340101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03452ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340102320 0x7f0340101c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39294/0 (socket says 172.21.15.92:39294) 2024-03-19T01:57:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03452ee640 1 -- 172.21.15.92:0/3907235763 learned_addr learned my addr 172.21.15.92:0/3907235763 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0345aef640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0340107580 0x7f0340100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f0344aed640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340104c50 0x7f03401002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03452ee640 1 -- 172.21.15.92:0/3907235763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0340107580 msgr2=0x7f0340100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03452ee640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0340107580 0x7f0340100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03452ee640 1 -- 172.21.15.92:0/3907235763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340104c50 msgr2=0x7f03401002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03452ee640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340104c50 0x7f03401002d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03452ee640 1 -- 172.21.15.92:0/3907235763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03401010a0 con 0x7f0340102320 2024-03-19T01:57:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03452ee640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340102320 0x7f0340101c90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e9f0 tx=0x7f033c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03367fc640 1 -- 172.21.15.92:0/3907235763 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c00cd80 con 0x7f0340102320 2024-03-19T01:57:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.940+0000 7f03367fc640 1 -- 172.21.15.92:0/3907235763 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f033c005500 con 0x7f0340102320 2024-03-19T01:57:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.941+0000 7f0347552640 1 -- 172.21.15.92:0/3907235763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03401a1bf0 con 0x7f0340102320 2024-03-19T01:57:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.941+0000 7f03367fc640 1 -- 172.21.15.92:0/3907235763 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c010690 con 0x7f0340102320 2024-03-19T01:57:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.941+0000 7f0347552640 1 -- 172.21.15.92:0/3907235763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03401a2140 con 0x7f0340102320 2024-03-19T01:57:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.943+0000 7f03367fc640 1 -- 172.21.15.92:0/3907235763 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f033c0046c0 con 0x7f0340102320 2024-03-19T01:57:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.943+0000 7f034a210740 1 Processor -- start 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.943+0000 7f034a210740 1 -- start start 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.943+0000 7f034a210740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d349c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.943+0000 7f034a210740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.943+0000 7f034a210740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d34a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.943+0000 7f034a210740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e9d3400be0 con 0x55e9d349caf0 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.943+0000 7f034a210740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e9d33a9960 con 0x55e9d349c100 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.943+0000 7f034a210740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e9d33fcc80 con 0x55e9d34a1c60 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03367fc640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f032c0a2920 0x7f032c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03452ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f0344aed640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d34a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03452ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39310/0 (socket says 172.21.15.92:39310) 2024-03-19T01:57:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03452ee640 1 -- 172.21.15.92:0/2764477398 learned_addr learned my addr 172.21.15.92:0/2764477398 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f0345aef640 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d349c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f0345aef640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f032c0a2920 0x7f032c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03367fc640 1 -- 172.21.15.92:0/3907235763 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f033c0cedf0 con 0x7f0340102320 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03452ee640 1 -- 172.21.15.92:0/2764477398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 msgr2=0x55e9d34a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03452ee640 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d34a40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03452ee640 1 -- 172.21.15.92:0/2764477398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 msgr2=0x55e9d349c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03452ee640 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d349c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03452ee640 1 -- 172.21.15.92:0/2764477398 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9d33a0d20 con 0x55e9d349caf0 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f0344aed640 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d34a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f0345aef640 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d349c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f03452ee640 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a15f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e9c0 tx=0x7f033c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32bc5d430e57041 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f0345aef640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f032c0a2920 0x7f032c0a4de0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0338005e30 tx=0x7f0338032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f032b7fe640 1 -- 172.21.15.92:0/2764477398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c09ba00 con 0x55e9d349caf0 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f032b7fe640 1 -- 172.21.15.92:0/2764477398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f033c09bea0 con 0x55e9d349caf0 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.944+0000 7f032b7fe640 1 -- 172.21.15.92:0/2764477398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c09b7a0 con 0x55e9d349caf0 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 -- 172.21.15.92:0/2764477398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 msgr2=0x55e9d34a15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a15f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e9c0 tx=0x7f033c004a40 comp rx=0 tx=0).stop 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 -- 172.21.15.92:0/2764477398 shutdown_connections 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d34a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a15f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 --2- 172.21.15.92:0/2764477398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d349c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 -- 172.21.15.92:0/2764477398 >> 172.21.15.92:0/2764477398 conn(0x55e9d3497670 msgr2=0x55e9d3497a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 -- 172.21.15.92:0/2764477398 shutdown_connections 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 -- 172.21.15.92:0/2764477398 wait complete. 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.945+0000 7f034a210740 1 -- start start 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f034a210740 1 Processor -- start 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f034a210740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d34a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f034a210740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f034a210740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d34ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f034a210740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e9d34a4bb0 con 0x55e9d349caf0 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f034a210740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e9d33bcdb0 con 0x55e9d349c100 2024-03-19T01:57:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f034a210740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e9d34a4d30 con 0x55e9d34a1c60 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f034a210740 10 client.0 _ll_register_callbacks cb 0x55e9d33c1e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f0344aed640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f0344aed640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39316/0 (socket says 172.21.15.92:39316) 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f0344aed640 1 -- 172.21.15.92:0/492489426 learned_addr learned my addr 172.21.15.92:0/492489426 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f0345aef640 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d34ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f03452ee640 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d34a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f0344aed640 1 -- 172.21.15.92:0/492489426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 msgr2=0x55e9d34ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f0344aed640 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d34ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f0344aed640 1 -- 172.21.15.92:0/492489426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 msgr2=0x55e9d34a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f0344aed640 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d34a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f0344aed640 1 -- 172.21.15.92:0/492489426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9d33bbc60 con 0x55e9d349caf0 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.946+0000 7f03452ee640 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d34a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:57:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f0344aed640 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a65b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f03300126e0 tx=0x7f0330012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b8565fef7ee164 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f0328ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f0328ff9640 1 -- 172.21.15.92:0/492489426 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330012190 con 0x55e9d349caf0 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f0328ff9640 1 -- 172.21.15.92:0/492489426 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f033000f8c0 con 0x55e9d349caf0 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f034a210740 1 -- 172.21.15.92:0/492489426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f033c09bea0 con 0x55e9d349caf0 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f0328ff9640 1 -- 172.21.15.92:0/492489426 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f033000fac0 con 0x55e9d349caf0 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f034a210740 10 client.15708 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f034a210740 1 -- 172.21.15.92:0/492489426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9d3115210 con 0x55e9d349caf0 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f0328ff9640 1 -- 172.21.15.92:0/492489426 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f033c09bea0 con 0x55e9d349caf0 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f0328ff9640 1 -- 172.21.15.92:0/492489426 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0330010ca0 con 0x55e9d349caf0 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f034a210740 10 client.15708 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f034a210740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f034a210740 20 client.15708 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f034a210740 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e9d34b1f00 0x55e9d34b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f034a210740 4 client.15708 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.947+0000 7f034a210740 1 -- 172.21.15.92:0/492489426 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9d340e120 con 0x55e9d34b1f00 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.948+0000 7f03452ee640 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e9d34b1f00 0x55e9d34b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.948+0000 7f03452ee640 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e9d34b1f00 0x55e9d34b43a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd1c2feb56ca404 server_cookie=598e78754ad454ed in_seq=0 out_seq=0 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.948+0000 7f0328ff9640 10 client.15708 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.948+0000 7f0328ff9640 1 -- 172.21.15.92:0/492489426 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e9d340e120 con 0x55e9d34b1f00 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f0328ff9640 10 client.15708 handle_command_reply: tid=0 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 client.15708 shutdown 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 -- 172.21.15.92:0/492489426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 msgr2=0x55e9d34a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a65b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f03300126e0 tx=0x7f0330012ba0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 -- 172.21.15.92:0/492489426 shutdown_connections 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e9d34b1f00 0x55e9d34b43a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d34ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349caf0 0x55e9d34a65b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 --2- 172.21.15.92:0/492489426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349c100 0x55e9d34a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 -- 172.21.15.92:0/492489426 >> 172.21.15.92:0/492489426 conn(0x55e9d3497670 msgr2=0x55e9d34988c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f0328ff9640 0 client.15708 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 -- 172.21.15.92:0/492489426 shutdown_connections 2024-03-19T01:57:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 1 -- 172.21.15.92:0/492489426 wait complete. 2024-03-19T01:57:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.949+0000 7f034a210740 20 client.15708 trim_cache size 0 max 16384 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f034a210740 1 Processor -- start 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f034a210740 1 -- start start 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f034a210740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349c100 0x55e9d3496930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f034a210740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 0x55e9d34ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f034a210740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d349d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f034a210740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03401aaf20 con 0x55e9d349c100 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f034a210740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03401aca60 con 0x55e9d349caf0 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f034a210740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f03401ab5f0 con 0x55e9d34a1c60 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f03452ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349c100 0x55e9d3496930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f03452ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349c100 0x55e9d3496930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39332/0 (socket says 172.21.15.92:39332) 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f03452ee640 1 -- 172.21.15.92:0/1930944999 learned_addr learned my addr 172.21.15.92:0/1930944999 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f0345aef640 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d349d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.975+0000 7f0344aed640 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 0x55e9d34ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f03452ee640 1 -- 172.21.15.92:0/1930944999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 msgr2=0x55e9d349d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f03452ee640 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d349d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f03452ee640 1 -- 172.21.15.92:0/1930944999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 msgr2=0x55e9d34ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f03452ee640 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 0x55e9d34ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f03452ee640 1 -- 172.21.15.92:0/1930944999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9d31e1620 con 0x55e9d349c100 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f0344aed640 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 0x55e9d34ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f03452ee640 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349c100 0x55e9d3496930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f033c09be00 tx=0x7f033c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43853fb5341f552d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f032affd640 1 -- 172.21.15.92:0/1930944999 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c0b3040 con 0x55e9d349c100 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f032affd640 1 -- 172.21.15.92:0/1930944999 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f033c09bea0 con 0x55e9d349c100 2024-03-19T01:57:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f032affd640 1 -- 172.21.15.92:0/1930944999 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c09f8f0 con 0x55e9d349c100 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f034a210740 1 -- 172.21.15.92:0/1930944999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349c100 msgr2=0x55e9d3496930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f034a210740 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349c100 0x55e9d3496930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f033c09be00 tx=0x7f033c086040 comp rx=0 tx=0).stop 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f034a210740 1 -- 172.21.15.92:0/1930944999 shutdown_connections 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f034a210740 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d34a1c60 0x55e9d349d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f034a210740 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 0x55e9d34ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f034a210740 1 --2- 172.21.15.92:0/1930944999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d349c100 0x55e9d3496930 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.976+0000 7f034a210740 1 -- 172.21.15.92:0/1930944999 >> 172.21.15.92:0/1930944999 conn(0x55e9d3497670 msgr2=0x55e9d3496220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 -- 172.21.15.92:0/1930944999 shutdown_connections 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 -- 172.21.15.92:0/1930944999 wait complete. 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 -- start start 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 Processor -- start 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d349c100 0x55e9d34aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 0x55e9d34ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d34a1c60 0x55e9d34ac470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03401ac390 con 0x55e9d34a1c60 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03401abcc0 con 0x55e9d349caf0 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f03401a8f00 con 0x55e9d349c100 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f034a210740 10 client.0 _ll_register_callbacks cb 0x7f0330009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f0345aef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d34a1c60 0x55e9d34ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f0345aef640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d34a1c60 0x55e9d34ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39334/0 (socket says 172.21.15.92:39334) 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f0345aef640 1 -- 172.21.15.92:0/3337683276 learned_addr learned my addr 172.21.15.92:0/3337683276 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f03452ee640 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d349c100 0x55e9d34aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.977+0000 7f0344aed640 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 0x55e9d34ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0345aef640 1 -- 172.21.15.92:0/3337683276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d349c100 msgr2=0x55e9d34aaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0345aef640 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d349c100 0x55e9d34aaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0345aef640 1 -- 172.21.15.92:0/3337683276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 msgr2=0x55e9d34ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0345aef640 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 0x55e9d34ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0345aef640 1 -- 172.21.15.92:0/3337683276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9d34aca60 con 0x55e9d34a1c60 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0345aef640 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d34a1c60 0x55e9d34ac470 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0338009ca0 tx=0x7f03380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9c1b50bb1ab977 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0328ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0328ff9640 1 -- 172.21.15.92:0/3337683276 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338054070 con 0x55e9d34a1c60 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0328ff9640 1 -- 172.21.15.92:0/3337683276 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0338004450 con 0x55e9d34a1c60 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f034a210740 1 -- 172.21.15.92:0/3337683276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f033c09bea0 con 0x55e9d34a1c60 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0328ff9640 1 -- 172.21.15.92:0/3337683276 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0338039040 con 0x55e9d34a1c60 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f034a210740 10 client.15714 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f034a210740 1 -- 172.21.15.92:0/3337683276 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9d3518610 con 0x55e9d34a1c60 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0328ff9640 1 -- 172.21.15.92:0/3337683276 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f033c09bea0 con 0x55e9d34a1c60 2024-03-19T01:57:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.978+0000 7f0328ff9640 1 -- 172.21.15.92:0/3337683276 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0338005500 con 0x55e9d34a1c60 2024-03-19T01:57:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.979+0000 7f034a210740 10 client.15714 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.979+0000 7f034a210740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.979+0000 7f034a210740 20 client.15714 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.979+0000 7f034a210740 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e9d3519090 0x55e9d351b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.979+0000 7f034a210740 4 client.15714 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:57:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.979+0000 7f034a210740 1 -- 172.21.15.92:0/3337683276 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e9d3408470 con 0x55e9d3519090 2024-03-19T01:57:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.979+0000 7f03452ee640 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e9d3519090 0x55e9d351b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.979+0000 7f0328ff9640 10 client.15714 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.979+0000 7f03452ee640 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e9d3519090 0x55e9d351b4b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b6c45897523cd server_cookie=53cc44423ffb1f47 in_seq=0 out_seq=0 2024-03-19T01:57:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f0328ff9640 1 -- 172.21.15.92:0/3337683276 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e9d3408470 con 0x55e9d3519090 2024-03-19T01:57:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f0328ff9640 10 client.15714 handle_command_reply: tid=0 2024-03-19T01:57:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 client.15714 shutdown 2024-03-19T01:57:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 -- 172.21.15.92:0/3337683276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d34a1c60 msgr2=0x55e9d34ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d34a1c60 0x55e9d34ac470 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0338009ca0 tx=0x7f03380095c0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 -- 172.21.15.92:0/3337683276 shutdown_connections 2024-03-19T01:57:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e9d3519090 0x55e9d351b4b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e9d34a1c60 0x55e9d34ac470 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9d349caf0 0x55e9d34ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f0328ff9640 0 client.15714 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 --2- 172.21.15.92:0/3337683276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9d349c100 0x55e9d34aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 -- 172.21.15.92:0/3337683276 >> 172.21.15.92:0/3337683276 conn(0x55e9d3497670 msgr2=0x55e9d349a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 -- 172.21.15.92:0/3337683276 shutdown_connections 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 1 -- 172.21.15.92:0/3337683276 wait complete. 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f034a210740 20 client.15714 trim_cache size 0 max 16384 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:57:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:57:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f0347552640 1 -- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f032c0a2920 msgr2=0x7f032c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f0347552640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f032c0a2920 0x7f032c0a4de0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0338005e30 tx=0x7f0338032590 comp rx=0 tx=0).stop 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f0347552640 1 -- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340102320 msgr2=0x7f0340101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.986+0000 7f0347552640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340102320 0x7f0340101c90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f033c00e9f0 tx=0x7f033c00edf0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.987+0000 7f0347552640 1 -- 172.21.15.92:0/3907235763 shutdown_connections 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.987+0000 7f0347552640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f032c0a2920 0x7f032c0a4de0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.987+0000 7f0347552640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0340107580 0x7f0340100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.987+0000 7f0347552640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0340104c50 0x7f03401002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.987+0000 7f0347552640 1 --2- 172.21.15.92:0/3907235763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0340102320 0x7f0340101c90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.987+0000 7f0347552640 1 -- 172.21.15.92:0/3907235763 >> 172.21.15.92:0/3907235763 conn(0x7f03400fbcb0 msgr2=0x7f0340102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.987+0000 7f0347552640 1 -- 172.21.15.92:0/3907235763 shutdown_connections 2024-03-19T01:57:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:43.987+0000 7f0347552640 1 -- 172.21.15.92:0/3907235763 wait complete. 2024-03-19T01:57:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:44 smithi159 ceph-mon[67110]: pgmap v499: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:57:45.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:44 smithi092 ceph-mon[62574]: pgmap v499: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:57:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:44 smithi144 ceph-mon[67199]: pgmap v499: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:57:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:46 smithi159 ceph-mon[67110]: pgmap v500: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-03-19T01:57:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:46 smithi092 ceph-mon[62574]: pgmap v500: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-03-19T01:57:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:46 smithi144 ceph-mon[67199]: pgmap v500: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-03-19T01:57:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:48 smithi159 ceph-mon[67110]: pgmap v501: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-03-19T01:57:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:48 smithi092 ceph-mon[62574]: pgmap v501: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-03-19T01:57:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:48 smithi144 ceph-mon[67199]: pgmap v501: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-03-19T01:57:50.584 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:189/600: open fd = 3 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:190/600: open fd = 3 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:191/600: open fd = 3 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:192/600: open fd = 3 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.585 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:193/600: open fd = 3 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:194/600: open fd = 3 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:195/600: open fd = 3 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:196/600: open fd = 3 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.586 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:197/600: open fd = 3 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:198/600: open fd = 3 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:199/600: open fd = 3 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:200/600: open fd = 3 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.587 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:201/600: open fd = 3 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:202/600: open fd = 3 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:203/600: open fd = 3 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:204/600: open fd = 3 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.588 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:205/600: open fd = 3 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:206/600: open fd = 3 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:207/600: open fd = 3 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.589 INFO:tasks.workunit.client.0.smithi092.stdout:208/600: open fd = 3 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:209/600: open fd = 3 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:210/600: open fd = 3 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:211/600: open fd = 3 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.590 INFO:tasks.workunit.client.0.smithi092.stdout:212/600: open fd = 3 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:213/600: open fd = 3 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:214/600: open fd = 3 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.591 INFO:tasks.workunit.client.0.smithi092.stdout:215/600: open fd = 3 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:216/600: open fd = 3 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:217/600: open fd = 3 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.592 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:218/600: open fd = 3 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:219/600: open fd = 3 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:220/600: open fd = 3 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.593 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:221/600: open fd = 3 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:222/600: open fd = 3 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:223/600: open fd = 3 2024-03-19T01:57:50.594 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:224/600: open fd = 3 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:225/600: open fd = 3 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:226/600: open fd = 3 2024-03-19T01:57:50.595 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:227/600: open fd = 3 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:228/600: open fd = 3 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:229/600: open fd = 3 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.596 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:230/600: open fd = 3 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:231/600: open fd = 3 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:232/600: open fd = 3 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:233/600: open fd = 3 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.597 INFO:tasks.workunit.client.0.smithi092.stdout:234/600: open fd = 3 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:235/600: open fd = 3 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:236/600: open fd = 3 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:237/600: open fd = 3 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.598 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:238/600: open fd = 3 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:239/600: open fd = 3 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:240/600: open fd = 3 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.599 INFO:tasks.workunit.client.0.smithi092.stdout:241/600: open fd = 3 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:242/600: open fd = 3 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:243/600: open fd = 3 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.600 INFO:tasks.workunit.client.0.smithi092.stdout:244/600: open fd = 3 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:245/600: open fd = 3 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:246/600: open fd = 3 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:247/600: open fd = 3 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.601 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:248/600: open fd = 3 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:249/600: open fd = 3 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:250/600: open fd = 3 2024-03-19T01:57:50.602 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.603 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:57:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:50 smithi159 ceph-mon[67110]: pgmap v502: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:57:51.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:50 smithi092 ceph-mon[62574]: pgmap v502: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:57:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:50 smithi144 ceph-mon[67199]: pgmap v502: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T01:57:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:52 smithi159 ceph-mon[67110]: pgmap v503: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:53.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:52 smithi092 ceph-mon[62574]: pgmap v503: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:52 smithi144 ceph-mon[67199]: pgmap v503: 97 pgs: 97 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:54 smithi159 ceph-mon[67110]: pgmap v504: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:55.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:54 smithi092 ceph-mon[62574]: pgmap v504: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:55.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:54 smithi144 ceph-mon[67199]: pgmap v504: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:57:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:56 smithi159 ceph-mon[67110]: pgmap v505: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:57:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:56 smithi092 ceph-mon[62574]: pgmap v505: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:57:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.075+0000 7f15d37c6640 1 -- 172.21.15.92:0/3941008112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc1022f0 msgr2=0x7f15cc108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.075+0000 7f15d37c6640 1 --2- 172.21.15.92:0/3941008112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc1022f0 0x7f15cc108b80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009ca0 tx=0x7f15c402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.075+0000 7f15d37c6640 1 -- 172.21.15.92:0/3941008112 shutdown_connections 2024-03-19T01:57:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.075+0000 7f15d37c6640 1 --2- 172.21.15.92:0/3941008112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc1022f0 0x7f15cc108b80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.075+0000 7f15d37c6640 1 --2- 172.21.15.92:0/3941008112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc101930 0x7f15cc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.075+0000 7f15d37c6640 1 --2- 172.21.15.92:0/3941008112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc100740 0x7f15cc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.075+0000 7f15d37c6640 1 -- 172.21.15.92:0/3941008112 >> 172.21.15.92:0/3941008112 conn(0x7f15cc0fbcb0 msgr2=0x7f15cc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 -- 172.21.15.92:0/3941008112 shutdown_connections 2024-03-19T01:57:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 -- 172.21.15.92:0/3941008112 wait complete. 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 Processor -- start 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 -- start start 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc100740 0x7f15cc199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc101930 0x7f15cc19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc1022f0 0x7f15cc1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc10c3c0 con 0x7f15cc101930 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc10c240 con 0x7f15cc1022f0 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d37c6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc10c540 con 0x7f15cc100740 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d0d61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc101930 0x7f15cc19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d0d61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc101930 0x7f15cc19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45332/0 (socket says 172.21.15.92:45332) 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d0d61640 1 -- 172.21.15.92:0/4095079007 learned_addr learned my addr 172.21.15.92:0/4095079007 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d1562640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc100740 0x7f15cc199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.076+0000 7f15d1d63640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc1022f0 0x7f15cc1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15d0d61640 1 -- 172.21.15.92:0/4095079007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc100740 msgr2=0x7f15cc199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15d0d61640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc100740 0x7f15cc199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15d0d61640 1 -- 172.21.15.92:0/4095079007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc1022f0 msgr2=0x7f15cc1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15d0d61640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc1022f0 0x7f15cc1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15d0d61640 1 -- 172.21.15.92:0/4095079007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc1a1880 con 0x7f15cc101930 2024-03-19T01:57:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15d0d61640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc101930 0x7f15cc19a1b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e9f0 tx=0x7f15c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15ca7fc640 1 -- 172.21.15.92:0/4095079007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c000cd80 con 0x7f15cc101930 2024-03-19T01:57:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15d37c6640 1 -- 172.21.15.92:0/4095079007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc1a1b70 con 0x7f15cc101930 2024-03-19T01:57:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15ca7fc640 1 -- 172.21.15.92:0/4095079007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c0010910 con 0x7f15cc101930 2024-03-19T01:57:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15ca7fc640 1 -- 172.21.15.92:0/4095079007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c00188f0 con 0x7f15cc101930 2024-03-19T01:57:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.077+0000 7f15d37c6640 1 -- 172.21.15.92:0/4095079007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc1a2110 con 0x7f15cc101930 2024-03-19T01:57:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.079+0000 7f15d63b1740 1 Processor -- start 2024-03-19T01:57:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d63b1740 1 -- start start 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db83520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83af0 0x561c0db885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db88c60 0x561c0db8b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d63b1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561c0dae7be0 con 0x561c0db83af0 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d63b1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c0da90960 con 0x561c0db88c60 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d63b1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561c0dae3c80 con 0x561c0db83100 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1562640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83af0 0x561c0db885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1d63640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db83520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1562640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83af0 0x561c0db885f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45336/0 (socket says 172.21.15.92:45336) 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1562640 1 -- 172.21.15.92:0/3191309223 learned_addr learned my addr 172.21.15.92:0/3191309223 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15ca7fc640 1 -- 172.21.15.92:0/4095079007 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f15c0010ac0 con 0x7f15cc101930 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1562640 1 -- 172.21.15.92:0/3191309223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 msgr2=0x561c0db83520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1562640 1 --2- 172.21.15.92:0/3191309223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db83520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1562640 1 -- 172.21.15.92:0/3191309223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db88c60 msgr2=0x561c0db8b0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1562640 1 --2- 172.21.15.92:0/3191309223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db88c60 0x561c0db8b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1562640 1 -- 172.21.15.92:0/3191309223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0da87d20 con 0x561c0db83af0 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1d63640 1 --2- 172.21.15.92:0/3191309223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db83520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15d1562640 1 --2- 172.21.15.92:0/3191309223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83af0 0x561c0db885f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009c60 tx=0x7f15bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e788cb7c6daf471f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15b37fe640 1 -- 172.21.15.92:0/3191309223 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc004220 con 0x561c0db83af0 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15b37fe640 1 -- 172.21.15.92:0/3191309223 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15bc0043c0 con 0x561c0db83af0 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.080+0000 7f15b37fe640 1 -- 172.21.15.92:0/3191309223 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0198a0 con 0x561c0db83af0 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15ca7fc640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f15b40a28a0 0x7f15b40a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15ca7fc640 1 -- 172.21.15.92:0/4095079007 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f15c00cee80 con 0x7f15cc101930 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d1d63640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f15b40a28a0 0x7f15b40a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d63b1740 1 -- 172.21.15.92:0/3191309223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83af0 msgr2=0x561c0db885f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d1d63640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f15b40a28a0 0x7f15b40a4d60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f15c4005e30 tx=0x7f15c4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d63b1740 1 --2- 172.21.15.92:0/3191309223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83af0 0x561c0db885f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009c60 tx=0x7f15bc011810 comp rx=0 tx=0).stop 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d63b1740 1 -- 172.21.15.92:0/3191309223 shutdown_connections 2024-03-19T01:57:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d63b1740 1 --2- 172.21.15.92:0/3191309223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db88c60 0x561c0db8b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d63b1740 1 --2- 172.21.15.92:0/3191309223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83af0 0x561c0db885f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d63b1740 1 --2- 172.21.15.92:0/3191309223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db83520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d63b1740 1 -- 172.21.15.92:0/3191309223 >> 172.21.15.92:0/3191309223 conn(0x561c0db7e670 msgr2=0x561c0db7ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d63b1740 1 -- 172.21.15.92:0/3191309223 shutdown_connections 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.081+0000 7f15d63b1740 1 -- 172.21.15.92:0/3191309223 wait complete. 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d63b1740 1 -- start start 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d63b1740 1 Processor -- start 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db8d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db8d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0db94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d63b1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561c0db8bc10 con 0x561c0db88c60 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d63b1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c0daa3db0 con 0x561c0db83af0 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d63b1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561c0db8bd90 con 0x561c0db83100 2024-03-19T01:57:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d63b1740 10 client.0 _ll_register_callbacks cb 0x561c0daa8e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d1d63640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0db94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d1d63640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0db94680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45344/0 (socket says 172.21.15.92:45344) 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d1d63640 1 -- 172.21.15.92:0/863460609 learned_addr learned my addr 172.21.15.92:0/863460609 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d0d61640 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db8d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.082+0000 7f15d1562640 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db8d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d1d63640 1 -- 172.21.15.92:0/863460609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 msgr2=0x561c0db8d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d1d63640 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db8d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d1d63640 1 -- 172.21.15.92:0/863460609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 msgr2=0x561c0db8d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d1d63640 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db8d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d1d63640 1 -- 172.21.15.92:0/863460609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0daa2c60 con 0x561c0db88c60 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d1562640 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db8d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d0d61640 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db8d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d1d63640 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0db94680 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009ca0 tx=0x7f15c40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15049c0f6f31624 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15b0ff9640 1 -- 172.21.15.92:0/863460609 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4050020 con 0x561c0db88c60 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15b0ff9640 1 -- 172.21.15.92:0/863460609 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f15c404c470 con 0x561c0db88c60 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d63b1740 1 -- 172.21.15.92:0/863460609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15bc0043c0 con 0x561c0db88c60 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15b0ff9640 1 -- 172.21.15.92:0/863460609 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f15c4039040 con 0x561c0db88c60 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d63b1740 10 client.15726 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d63b1740 1 -- 172.21.15.92:0/863460609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c0d7fc210 con 0x561c0db88c60 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15b0ff9640 1 -- 172.21.15.92:0/863460609 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15bc0043c0 con 0x561c0db88c60 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15b0ff9640 1 -- 172.21.15.92:0/863460609 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f15c40347f0 con 0x561c0db88c60 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d63b1740 10 client.15726 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d63b1740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d63b1740 20 client.15726 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.083+0000 7f15d63b1740 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561c0db99040 0x561c0db9b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.084+0000 7f15d63b1740 4 client.15726 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.084+0000 7f15d63b1740 1 -- 172.21.15.92:0/863460609 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c0daf5120 con 0x561c0db99040 2024-03-19T01:57:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.084+0000 7f15d1562640 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561c0db99040 0x561c0db9b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.084+0000 7f15b0ff9640 10 client.15726 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.084+0000 7f15d1562640 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561c0db99040 0x561c0db9b430 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fea043a59d080fb server_cookie=424cd673fbf4a423 in_seq=0 out_seq=0 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15b0ff9640 1 -- 172.21.15.92:0/863460609 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c0daf5120 con 0x561c0db99040 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15b0ff9640 10 client.15726 handle_command_reply: tid=0 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 client.15726 shutdown 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 -- 172.21.15.92:0/863460609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 msgr2=0x561c0db94680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0db94680 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009ca0 tx=0x7f15c40095c0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 -- 172.21.15.92:0/863460609 shutdown_connections 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561c0db99040 0x561c0db9b430 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0db94680 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db8d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15b0ff9640 0 client.15726 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 --2- 172.21.15.92:0/863460609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db8d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 -- 172.21.15.92:0/863460609 >> 172.21.15.92:0/863460609 conn(0x561c0db7e670 msgr2=0x561c0db7f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 -- 172.21.15.92:0/863460609 shutdown_connections 2024-03-19T01:57:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 1 -- 172.21.15.92:0/863460609 wait complete. 2024-03-19T01:57:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.085+0000 7f15d63b1740 20 client.15726 trim_cache size 0 max 16384 2024-03-19T01:57:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:56 smithi144 ceph-mon[67199]: pgmap v505: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:57:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d63b1740 1 Processor -- start 2024-03-19T01:57:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d63b1740 1 -- start start 2024-03-19T01:57:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83100 0x561c0db8d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83af0 0x561c0db8d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db88c60 0x561c0db94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d63b1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1aad00 con 0x561c0db83100 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d63b1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1ac840 con 0x561c0db88c60 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d63b1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1ab3d0 con 0x561c0db83af0 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d1562640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83100 0x561c0db8d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d1562640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83100 0x561c0db8d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45346/0 (socket says 172.21.15.92:45346) 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d1562640 1 -- 172.21.15.92:0/284611755 learned_addr learned my addr 172.21.15.92:0/284611755 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d0d61640 1 --2- 172.21.15.92:0/284611755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83af0 0x561c0db8d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.110+0000 7f15d1d63640 1 --2- 172.21.15.92:0/284611755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db88c60 0x561c0db94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.111+0000 7f15d1562640 1 -- 172.21.15.92:0/284611755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83af0 msgr2=0x561c0db8d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.111+0000 7f15d1562640 1 --2- 172.21.15.92:0/284611755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83af0 0x561c0db8d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.111+0000 7f15d1562640 1 -- 172.21.15.92:0/284611755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db88c60 msgr2=0x561c0db94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.111+0000 7f15d1562640 1 --2- 172.21.15.92:0/284611755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db88c60 0x561c0db94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.111+0000 7f15d1562640 1 -- 172.21.15.92:0/284611755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0d8c8620 con 0x561c0db83100 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.111+0000 7f15d1562640 1 --2- 172.21.15.92:0/284611755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83100 0x561c0db8d100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009c30 tx=0x7f15bc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff594cb98216f22e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.111+0000 7f15b2ffd640 1 -- 172.21.15.92:0/284611755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc005370 con 0x561c0db83100 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.111+0000 7f15b2ffd640 1 -- 172.21.15.92:0/284611755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15bc005510 con 0x561c0db83100 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.111+0000 7f15b2ffd640 1 -- 172.21.15.92:0/284611755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc003670 con 0x561c0db83100 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 -- 172.21.15.92:0/284611755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83100 msgr2=0x561c0db8d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 --2- 172.21.15.92:0/284611755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83100 0x561c0db8d100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009c30 tx=0x7f15bc009680 comp rx=0 tx=0).stop 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 -- 172.21.15.92:0/284611755 shutdown_connections 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 --2- 172.21.15.92:0/284611755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db88c60 0x561c0db94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 --2- 172.21.15.92:0/284611755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83af0 0x561c0db8d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 --2- 172.21.15.92:0/284611755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db83100 0x561c0db8d100 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 -- 172.21.15.92:0/284611755 >> 172.21.15.92:0/284611755 conn(0x561c0db7e670 msgr2=0x561c0db7ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 -- 172.21.15.92:0/284611755 shutdown_connections 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 -- 172.21.15.92:0/284611755 wait complete. 2024-03-19T01:57:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 -- start start 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 Processor -- start 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db91350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db91890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0dbfd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1ac170 con 0x561c0db88c60 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1abaa0 con 0x561c0db83af0 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1a8ce0 con 0x561c0db83100 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.112+0000 7f15d63b1740 10 client.0 _ll_register_callbacks cb 0x7f15c4044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d0d61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db91890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d1562640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db91350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d1d63640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0dbfd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d0d61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db91890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:39992/0 (socket says 172.21.15.92:39992) 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d1562640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db91350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:53126/0 (socket says 172.21.15.92:53126) 2024-03-19T01:57:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d0d61640 1 -- 172.21.15.92:0/127221765 learned_addr learned my addr 172.21.15.92:0/127221765 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d1d63640 1 -- 172.21.15.92:0/127221765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 msgr2=0x561c0db91350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d1d63640 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db91350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d1d63640 1 -- 172.21.15.92:0/127221765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 msgr2=0x561c0db91890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d1d63640 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db91890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d1d63640 1 -- 172.21.15.92:0/127221765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0db94960 con 0x561c0db88c60 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d0d61640 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db91890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d1d63640 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0dbfd360 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006010 tx=0x7f15c4034ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea52cf181c2593c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15b0ff9640 1 -- 172.21.15.92:0/127221765 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4065060 con 0x561c0db88c60 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d63b1740 1 -- 172.21.15.92:0/127221765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15bc005510 con 0x561c0db88c60 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15b0ff9640 1 -- 172.21.15.92:0/127221765 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f15c404c880 con 0x561c0db88c60 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15b0ff9640 1 -- 172.21.15.92:0/127221765 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f15c4050040 con 0x561c0db88c60 2024-03-19T01:57:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d63b1740 10 client.15732 fetch_fsmap learned FSMap version 29 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.113+0000 7f15d63b1740 1 -- 172.21.15.92:0/127221765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c0dbfe8a0 con 0x561c0db88c60 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15b0ff9640 1 -- 172.21.15.92:0/127221765 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15bc005510 con 0x561c0db88c60 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15b0ff9640 1 -- 172.21.15.92:0/127221765 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f15c4060910 con 0x561c0db88c60 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15d63b1740 10 client.15732 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15d63b1740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15d63b1740 20 client.15732 populate_metadata read hostname 'smithi092' 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15d63b1740 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561c0dbff2a0 0x561c0dc016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15d63b1740 4 client.15732 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15d63b1740 1 -- 172.21.15.92:0/127221765 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c0daef470 con 0x561c0dbff2a0 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15d1562640 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561c0dbff2a0 0x561c0dc016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15b0ff9640 10 client.15732 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.114+0000 7f15d1562640 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561c0dbff2a0 0x561c0dc016c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c80c5c4626d79a9 server_cookie=13abd6409e4f45f9 in_seq=0 out_seq=0 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15b0ff9640 1 -- 172.21.15.92:0/127221765 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c0daef470 con 0x561c0dbff2a0 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15b0ff9640 10 client.15732 handle_command_reply: tid=0 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 client.15732 shutdown 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 -- 172.21.15.92:0/127221765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 msgr2=0x561c0dbfd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0dbfd360 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006010 tx=0x7f15c4034ab0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 -- 172.21.15.92:0/127221765 shutdown_connections 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561c0dbff2a0 0x561c0dc016c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c0db88c60 0x561c0dbfd360 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0db83af0 0x561c0db91890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15b0ff9640 0 client.15732 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 --2- 172.21.15.92:0/127221765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c0db83100 0x561c0db91350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 -- 172.21.15.92:0/127221765 >> 172.21.15.92:0/127221765 conn(0x561c0db7e670 msgr2=0x561c0db87cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 -- 172.21.15.92:0/127221765 shutdown_connections 2024-03-19T01:57:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 1 -- 172.21.15.92:0/127221765 wait complete. 2024-03-19T01:57:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.121+0000 7f15d63b1740 20 client.15732 trim_cache size 0 max 16384 2024-03-19T01:57:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:57:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:57:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:57:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:57:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 -- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f15b40a28a0 msgr2=0x7f15b40a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f15b40a28a0 0x7f15b40a4d60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f15c4005e30 tx=0x7f15c4032590 comp rx=0 tx=0).stop 2024-03-19T01:57:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 -- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc101930 msgr2=0x7f15cc19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:57:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc101930 0x7f15cc19a1b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e9f0 tx=0x7f15c000eeb0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 -- 172.21.15.92:0/4095079007 shutdown_connections 2024-03-19T01:57:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15cc1022f0 0x7f15cc1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15cc101930 0x7f15cc19a1b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f15b40a28a0 0x7f15b40a4d60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 --2- 172.21.15.92:0/4095079007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15cc100740 0x7f15cc199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:57:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 -- 172.21.15.92:0/4095079007 >> 172.21.15.92:0/4095079007 conn(0x7f15cc0fbcb0 msgr2=0x7f15cc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:57:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 -- 172.21.15.92:0/4095079007 shutdown_connections 2024-03-19T01:57:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:57:57.122+0000 7f15d37c6640 1 -- 172.21.15.92:0/4095079007 wait complete. 2024-03-19T01:57:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:57:58 smithi159 ceph-mon[67110]: pgmap v506: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:59.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:57:58 smithi092 ceph-mon[62574]: pgmap v506: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:57:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:57:58 smithi144 ceph-mon[67199]: pgmap v506: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:00 smithi159 ceph-mon[67110]: pgmap v507: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-19T01:58:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:00 smithi092 ceph-mon[62574]: pgmap v507: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-19T01:58:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:00 smithi144 ceph-mon[67199]: pgmap v507: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-19T01:58:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:02 smithi159 ceph-mon[67110]: pgmap v508: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T01:58:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:02 smithi092 ceph-mon[62574]: pgmap v508: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T01:58:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:02 smithi144 ceph-mon[67199]: pgmap v508: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T01:58:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:04 smithi159 ceph-mon[67110]: pgmap v509: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:58:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:04 smithi092 ceph-mon[62574]: pgmap v509: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:58:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:04 smithi144 ceph-mon[67199]: pgmap v509: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:58:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:06 smithi159 ceph-mon[67110]: pgmap v510: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:58:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:06 smithi092 ceph-mon[62574]: pgmap v510: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:58:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:06 smithi144 ceph-mon[67199]: pgmap v510: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:58:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:08 smithi159 ceph-mon[67110]: pgmap v511: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:58:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:08 smithi092 ceph-mon[62574]: pgmap v511: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:58:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:08 smithi144 ceph-mon[67199]: pgmap v511: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:58:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 -- 172.21.15.92:0/690813241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370102a60 msgr2=0x7ff370102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 --2- 172.21.15.92:0/690813241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370102a60 0x7ff370102ee0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff360009900 tx=0x7ff36002bc30 comp rx=0 tx=0).stop 2024-03-19T01:58:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 -- 172.21.15.92:0/690813241 shutdown_connections 2024-03-19T01:58:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 --2- 172.21.15.92:0/690813241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff370103420 0x7ff370109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 --2- 172.21.15.92:0/690813241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370102a60 0x7ff370102ee0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 --2- 172.21.15.92:0/690813241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff370101870 0x7ff370101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 -- 172.21.15.92:0/690813241 >> 172.21.15.92:0/690813241 conn(0x7ff3700fcde0 msgr2=0x7ff3700ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 -- 172.21.15.92:0/690813241 shutdown_connections 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 -- 172.21.15.92:0/690813241 wait complete. 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 Processor -- start 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.208+0000 7ff377e61640 1 -- start start 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff377e61640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370101870 0x7ff370197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff377e61640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff370102a60 0x7ff370197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff377e61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff370103420 0x7ff37019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff377e61640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010d4f0 con 0x7ff370101870 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff377e61640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010d370 con 0x7ff370103420 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff377e61640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff37010d670 con 0x7ff370102a60 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff375bfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370101870 0x7ff370197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff375bfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370101870 0x7ff370197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53298/0 (socket says 172.21.15.92:53298) 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff375bfd640 1 -- 172.21.15.92:0/2118163875 learned_addr learned my addr 172.21.15.92:0/2118163875 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff3753fc640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff370102a60 0x7ff370197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff3763fe640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff370103420 0x7ff37019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff375bfd640 1 -- 172.21.15.92:0/2118163875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff370102a60 msgr2=0x7ff370197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff375bfd640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff370102a60 0x7ff370197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff375bfd640 1 -- 172.21.15.92:0/2118163875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff370103420 msgr2=0x7ff37019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff375bfd640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff370103420 0x7ff37019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff375bfd640 1 -- 172.21.15.92:0/2118163875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37019f6a0 con 0x7ff370101870 2024-03-19T01:58:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff375bfd640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370101870 0x7ff370197a80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00e9c0 tx=0x7ff36c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff366ffd640 1 -- 172.21.15.92:0/2118163875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00cd80 con 0x7ff370101870 2024-03-19T01:58:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff366ffd640 1 -- 172.21.15.92:0/2118163875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff36c0035f0 con 0x7ff370101870 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff377e61640 1 -- 172.21.15.92:0/2118163875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37019f990 con 0x7ff370101870 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff366ffd640 1 -- 172.21.15.92:0/2118163875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c010690 con 0x7ff370101870 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.209+0000 7ff377e61640 1 -- 172.21.15.92:0/2118163875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37019fe60 con 0x7ff370101870 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.211+0000 7ff366ffd640 1 -- 172.21.15.92:0/2118163875 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff36c0037a0 con 0x7ff370101870 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff37aa51740 1 Processor -- start 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff37aa51740 1 -- start start 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff366ffd640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff35c0a28a0 0x7ff35c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0e100 0x55c50ba0e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba160f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff37aa51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c50b972be0 con 0x55c50ba13c60 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff37aa51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c50b91b960 con 0x55c50ba0e100 2024-03-19T01:58:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff37aa51740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c50b96ec80 con 0x55c50ba0eaf0 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff3763fe640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff35c0a28a0 0x7ff35c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.212+0000 7ff375bfd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0e100 0x55c50ba0e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff3753fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff366ffd640 1 -- 172.21.15.92:0/2118163875 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff36c0cedb0 con 0x7ff370101870 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff375bfd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0e100 0x55c50ba0e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:47260/0 (socket says 172.21.15.92:47260) 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff375bfd640 1 -- 172.21.15.92:0/3575280628 learned_addr learned my addr 172.21.15.92:0/3575280628 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff3763fe640 1 --2- 172.21.15.92:0/3575280628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff3753fc640 1 -- 172.21.15.92:0/3575280628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0e100 msgr2=0x55c50ba0e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff3753fc640 1 --2- 172.21.15.92:0/3575280628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0e100 0x55c50ba0e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff3753fc640 1 -- 172.21.15.92:0/3575280628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 msgr2=0x55c50ba160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff3753fc640 1 --2- 172.21.15.92:0/3575280628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff3753fc640 1 -- 172.21.15.92:0/3575280628 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50b912d20 con 0x55c50ba0eaf0 2024-03-19T01:58:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff3763fe640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff35c0a28a0 0x7ff35c0a4d60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff368005fd0 tx=0x7ff368009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff3753fc640 1 --2- 172.21.15.92:0/3575280628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba135f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff360009230 tx=0x7ff360009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a173404aa5a7ef server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff35bfff640 1 -- 172.21.15.92:0/3575280628 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36002cb80 con 0x55c50ba0eaf0 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff35bfff640 1 -- 172.21.15.92:0/3575280628 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff360005040 con 0x55c50ba0eaf0 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.213+0000 7ff35bfff640 1 -- 172.21.15.92:0/3575280628 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360004920 con 0x55c50ba0eaf0 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 -- 172.21.15.92:0/3575280628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 msgr2=0x55c50ba135f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 --2- 172.21.15.92:0/3575280628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba135f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff360009230 tx=0x7ff360009560 comp rx=0 tx=0).stop 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 -- 172.21.15.92:0/3575280628 shutdown_connections 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 --2- 172.21.15.92:0/3575280628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 --2- 172.21.15.92:0/3575280628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba135f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 --2- 172.21.15.92:0/3575280628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0e100 0x55c50ba0e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 -- 172.21.15.92:0/3575280628 >> 172.21.15.92:0/3575280628 conn(0x55c50ba09670 msgr2=0x55c50ba09a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 -- 172.21.15.92:0/3575280628 shutdown_connections 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 -- 172.21.15.92:0/3575280628 wait complete. 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 -- start start 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.214+0000 7ff37aa51740 1 Processor -- start 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 0x55c50ba18100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0eaf0 0x55c50ba18640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 0x55c50ba1f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff37aa51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c50ba16bb0 con 0x55c50ba0eaf0 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff37aa51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c50b92edb0 con 0x55c50ba13c60 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff37aa51740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c50ba16d30 con 0x55c50ba0e100 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff37aa51740 10 client.0 _ll_register_callbacks cb 0x55c50b933e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:58:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3753fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0eaf0 0x55c50ba18640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3753fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0eaf0 0x55c50ba18640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53320/0 (socket says 172.21.15.92:53320) 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3753fc640 1 -- 172.21.15.92:0/2786741643 learned_addr learned my addr 172.21.15.92:0/2786741643 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff375bfd640 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 0x55c50ba18100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3763fe640 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 0x55c50ba1f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3753fc640 1 -- 172.21.15.92:0/2786741643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 msgr2=0x55c50ba18100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3753fc640 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 0x55c50ba18100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3753fc640 1 -- 172.21.15.92:0/2786741643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 msgr2=0x55c50ba1f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3753fc640 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 0x55c50ba1f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3753fc640 1 -- 172.21.15.92:0/2786741643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50b92dc60 con 0x55c50ba0eaf0 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3753fc640 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0eaf0 0x55c50ba18640 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff360009900 tx=0x7ff36002cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b63d49140cc738 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.215+0000 7ff3597fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff3597fa640 1 -- 172.21.15.92:0/2786741643 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360005400 con 0x55c50ba0eaf0 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff3597fa640 1 -- 172.21.15.92:0/2786741643 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff3600046c0 con 0x55c50ba0eaf0 2024-03-19T01:58:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff37aa51740 1 -- 172.21.15.92:0/2786741643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff360005040 con 0x55c50ba0eaf0 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff3597fa640 1 -- 172.21.15.92:0/2786741643 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff3600048c0 con 0x55c50ba0eaf0 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff37aa51740 10 client.15744 fetch_fsmap learned FSMap version 29 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff37aa51740 1 -- 172.21.15.92:0/2786741643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c50b687210 con 0x55c50ba0eaf0 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff3597fa640 1 -- 172.21.15.92:0/2786741643 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff360005040 con 0x55c50ba0eaf0 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff3597fa640 1 -- 172.21.15.92:0/2786741643 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff360003a60 con 0x55c50ba0eaf0 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff37aa51740 10 client.15744 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff37aa51740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff37aa51740 20 client.15744 populate_metadata read hostname 'smithi092' 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c50ba23b00 0x55c50ba25ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff37aa51740 4 client.15744 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff37aa51740 1 -- 172.21.15.92:0/2786741643 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50b980120 con 0x55c50ba23b00 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.216+0000 7ff375bfd640 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c50ba23b00 0x55c50ba25ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.217+0000 7ff3597fa640 10 client.15744 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.217+0000 7ff375bfd640 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c50ba23b00 0x55c50ba25ef0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14164c11b75f8976 server_cookie=b7f06400aeac3528 in_seq=0 out_seq=0 2024-03-19T01:58:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.217+0000 7ff3597fa640 1 -- 172.21.15.92:0/2786741643 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c50b980120 con 0x55c50ba23b00 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.217+0000 7ff3597fa640 10 client.15744 handle_command_reply: tid=0 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.217+0000 7ff37aa51740 1 client.15744 shutdown 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 -- 172.21.15.92:0/2786741643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0eaf0 msgr2=0x55c50ba18640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0eaf0 0x55c50ba18640 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff360009900 tx=0x7ff36002cbb0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 -- 172.21.15.92:0/2786741643 shutdown_connections 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c50ba23b00 0x55c50ba25ef0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 0x55c50ba1f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0eaf0 0x55c50ba18640 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff3597fa640 0 client.15744 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2786741643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 0x55c50ba18100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 -- 172.21.15.92:0/2786741643 >> 172.21.15.92:0/2786741643 conn(0x55c50ba09670 msgr2=0x55c50ba0a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 -- 172.21.15.92:0/2786741643 shutdown_connections 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 1 -- 172.21.15.92:0/2786741643 wait complete. 2024-03-19T01:58:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.218+0000 7ff37aa51740 20 client.15744 trim_cache size 0 max 16384 2024-03-19T01:58:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.232+0000 7ff37aa51740 1 Processor -- start 2024-03-19T01:58:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.232+0000 7ff37aa51740 1 -- start start 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0e100 0x55c50ba08930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 0x55c50ba0f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff37aa51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a6940 con 0x55c50ba0e100 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff37aa51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a8480 con 0x55c50ba13c60 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff37aa51740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a7010 con 0x55c50ba0eaf0 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff375bfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0e100 0x55c50ba08930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff375bfd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0e100 0x55c50ba08930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53328/0 (socket says 172.21.15.92:53328) 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff375bfd640 1 -- 172.21.15.92:0/2074613944 learned_addr learned my addr 172.21.15.92:0/2074613944 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff3753fc640 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff3763fe640 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 0x55c50ba0f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff375bfd640 1 -- 172.21.15.92:0/2074613944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 msgr2=0x55c50ba1f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff375bfd640 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff375bfd640 1 -- 172.21.15.92:0/2074613944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 msgr2=0x55c50ba0f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff375bfd640 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 0x55c50ba0f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff375bfd640 1 -- 172.21.15.92:0/2074613944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50b753620 con 0x55c50ba0e100 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff3763fe640 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 0x55c50ba0f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff3753fc640 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff375bfd640 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0e100 0x55c50ba08930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff36c09b460 tx=0x7ff36c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c05a3855c4fe9ab server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff35b7fe640 1 -- 172.21.15.92:0/2074613944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c09b760 con 0x55c50ba0e100 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff35b7fe640 1 -- 172.21.15.92:0/2074613944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff36c09f8b0 con 0x55c50ba0e100 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.233+0000 7ff35b7fe640 1 -- 172.21.15.92:0/2074613944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c003b10 con 0x55c50ba0e100 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 -- 172.21.15.92:0/2074613944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0e100 msgr2=0x55c50ba08930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0e100 0x55c50ba08930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff36c09b460 tx=0x7ff36c086040 comp rx=0 tx=0).stop 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 -- 172.21.15.92:0/2074613944 shutdown_connections 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba13c60 0x55c50ba0f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 --2- 172.21.15.92:0/2074613944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba0e100 0x55c50ba08930 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 -- 172.21.15.92:0/2074613944 >> 172.21.15.92:0/2074613944 conn(0x55c50ba09670 msgr2=0x55c50ba08220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 -- 172.21.15.92:0/2074613944 shutdown_connections 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 -- 172.21.15.92:0/2074613944 wait complete. 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 -- start start 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.234+0000 7ff37aa51740 1 Processor -- start 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 0x55c50ba1f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff37aa51740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba1d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff37aa51740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a7db0 con 0x55c50ba13c60 2024-03-19T01:58:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff37aa51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a76e0 con 0x55c50ba0eaf0 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff37aa51740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff3701a4920 con 0x55c50ba0e100 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff37aa51740 10 client.0 _ll_register_callbacks cb 0x7ff36801a850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3763fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba1d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3753fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3763fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba1d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53330/0 (socket says 172.21.15.92:53330) 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff375bfd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 0x55c50ba1f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3763fe640 1 -- 172.21.15.92:0/65230678 learned_addr learned my addr 172.21.15.92:0/65230678 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3763fe640 1 -- 172.21.15.92:0/65230678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 msgr2=0x55c50ba1f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3763fe640 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 0x55c50ba1f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3763fe640 1 -- 172.21.15.92:0/65230678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0eaf0 msgr2=0x55c50ba1f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3763fe640 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3763fe640 1 -- 172.21.15.92:0/65230678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50ba1e030 con 0x55c50ba13c60 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3753fc640 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3763fe640 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba1d800 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff36801af30 tx=0x7ff368009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09fc255aea29f2a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3597fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3597fa640 1 -- 172.21.15.92:0/65230678 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368004b20 con 0x55c50ba13c60 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff37aa51740 1 -- 172.21.15.92:0/65230678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff36c09f8b0 con 0x55c50ba13c60 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.235+0000 7ff3597fa640 1 -- 172.21.15.92:0/65230678 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff368004cc0 con 0x55c50ba13c60 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff3597fa640 1 -- 172.21.15.92:0/65230678 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff368005020 con 0x55c50ba13c60 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff37aa51740 10 client.15750 fetch_fsmap learned FSMap version 29 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff37aa51740 1 -- 172.21.15.92:0/65230678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c50ba8a9e0 con 0x55c50ba13c60 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff3597fa640 1 -- 172.21.15.92:0/65230678 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff36c09f8b0 con 0x55c50ba13c60 2024-03-19T01:58:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff3597fa640 1 -- 172.21.15.92:0/65230678 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff368027590 con 0x55c50ba13c60 2024-03-19T01:58:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff37aa51740 10 client.15750 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:58:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff37aa51740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:58:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff37aa51740 20 client.15750 populate_metadata read hostname 'smithi092' 2024-03-19T01:58:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff37aa51740 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c50ba8b3c0 0x55c50ba8d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff37aa51740 4 client.15750 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:58:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff37aa51740 1 -- 172.21.15.92:0/65230678 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c50b97a470 con 0x55c50ba8b3c0 2024-03-19T01:58:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.236+0000 7ff375bfd640 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c50ba8b3c0 0x55c50ba8d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.237+0000 7ff3597fa640 10 client.15750 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.237+0000 7ff375bfd640 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c50ba8b3c0 0x55c50ba8d7e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697b72d2a9729cca server_cookie=9887614eddeff5d6 in_seq=0 out_seq=0 2024-03-19T01:58:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff3597fa640 1 -- 172.21.15.92:0/65230678 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c50b97a470 con 0x55c50ba8b3c0 2024-03-19T01:58:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff3597fa640 10 client.15750 handle_command_reply: tid=0 2024-03-19T01:58:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 client.15750 shutdown 2024-03-19T01:58:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 -- 172.21.15.92:0/65230678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 msgr2=0x55c50ba1d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba1d800 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff36801af30 tx=0x7ff368009500 comp rx=0 tx=0).stop 2024-03-19T01:58:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 -- 172.21.15.92:0/65230678 shutdown_connections 2024-03-19T01:58:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c50ba8b3c0 0x55c50ba8d7e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c50ba13c60 0x55c50ba1d800 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff3597fa640 0 client.15750 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c50ba0eaf0 0x55c50ba1f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 --2- 172.21.15.92:0/65230678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c50ba0e100 0x55c50ba1f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 -- 172.21.15.92:0/65230678 >> 172.21.15.92:0/65230678 conn(0x55c50ba09670 msgr2=0x55c50ba0c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 -- 172.21.15.92:0/65230678 shutdown_connections 2024-03-19T01:58:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 1 -- 172.21.15.92:0/65230678 wait complete. 2024-03-19T01:58:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.243+0000 7ff37aa51740 20 client.15750 trim_cache size 0 max 16384 2024-03-19T01:58:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:58:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:58:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:58:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:58:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 -- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff35c0a28a0 msgr2=0x7ff35c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff35c0a28a0 0x7ff35c0a4d60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff368005fd0 tx=0x7ff368009f10 comp rx=0 tx=0).stop 2024-03-19T01:58:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 -- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370101870 msgr2=0x7ff370197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370101870 0x7ff370197a80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00e9c0 tx=0x7ff36c00ee80 comp rx=0 tx=0).stop 2024-03-19T01:58:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 -- 172.21.15.92:0/2118163875 shutdown_connections 2024-03-19T01:58:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff370103420 0x7ff37019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff35c0a28a0 0x7ff35c0a4d60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff370102a60 0x7ff370197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 --2- 172.21.15.92:0/2118163875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff370101870 0x7ff370197a80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 -- 172.21.15.92:0/2118163875 >> 172.21.15.92:0/2118163875 conn(0x7ff3700fcde0 msgr2=0x7ff370105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 -- 172.21.15.92:0/2118163875 shutdown_connections 2024-03-19T01:58:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:10.244+0000 7ff377e61640 1 -- 172.21.15.92:0/2118163875 wait complete. 2024-03-19T01:58:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:10 smithi092 ceph-mon[62574]: pgmap v512: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:10 smithi144 ceph-mon[67199]: pgmap v512: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:11.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:10 smithi159 ceph-mon[67110]: pgmap v512: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:12.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:11 smithi092 ceph-mon[62574]: pgmap v513: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T01:58:12.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:11 smithi144 ceph-mon[67199]: pgmap v513: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T01:58:12.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:11 smithi159 ceph-mon[67110]: pgmap v513: 97 pgs: 97 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T01:58:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:14 smithi159 ceph-mon[67110]: pgmap v514: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T01:58:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:14 smithi092 ceph-mon[62574]: pgmap v514: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T01:58:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:14 smithi144 ceph-mon[67199]: pgmap v514: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T01:58:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:16 smithi159 ceph-mon[67110]: pgmap v515: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:16 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:58:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:16 smithi092 ceph-mon[62574]: pgmap v515: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:16 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:58:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:16 smithi144 ceph-mon[67199]: pgmap v515: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:16 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:58:17.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:17 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:58:17.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:17 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:58:17.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:17 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:58:18.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:17 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:58:18.063 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:17 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:58:18.063 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:17 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:58:18.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:17 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:58:18.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:17 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:58:18.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:17 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:58:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:18 smithi159 ceph-mon[67110]: pgmap v516: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:58:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:18 smithi092 ceph-mon[62574]: pgmap v516: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:58:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:18 smithi144 ceph-mon[67199]: pgmap v516: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:58:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:20 smithi159 ceph-mon[67110]: pgmap v517: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:58:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:20 smithi092 ceph-mon[62574]: pgmap v517: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:58:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:20 smithi144 ceph-mon[67199]: pgmap v517: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:58:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:22 smithi159 ceph-mon[67110]: pgmap v518: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:22 smithi092 ceph-mon[62574]: pgmap v518: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:22 smithi144 ceph-mon[67199]: pgmap v518: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.330+0000 7fcf38f52640 1 -- 172.21.15.92:0/238968752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf34101930 msgr2=0x7fcf34101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.330+0000 7fcf38f52640 1 --2- 172.21.15.92:0/238968752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf34101930 0x7fcf34101db0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009900 tx=0x7fcf1c02bc30 comp rx=0 tx=0).stop 2024-03-19T01:58:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.330+0000 7fcf38f52640 1 -- 172.21.15.92:0/238968752 shutdown_connections 2024-03-19T01:58:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.330+0000 7fcf38f52640 1 --2- 172.21.15.92:0/238968752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf341022f0 0x7fcf34108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.330+0000 7fcf38f52640 1 --2- 172.21.15.92:0/238968752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf34101930 0x7fcf34101db0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.330+0000 7fcf38f52640 1 --2- 172.21.15.92:0/238968752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf34100740 0x7fcf34100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.330+0000 7fcf38f52640 1 -- 172.21.15.92:0/238968752 >> 172.21.15.92:0/238968752 conn(0x7fcf340fbcb0 msgr2=0x7fcf340fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.331+0000 7fcf38f52640 1 -- 172.21.15.92:0/238968752 shutdown_connections 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.331+0000 7fcf38f52640 1 -- 172.21.15.92:0/238968752 wait complete. 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.331+0000 7fcf38f52640 1 Processor -- start 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.331+0000 7fcf38f52640 1 -- start start 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.331+0000 7fcf38f52640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf34100740 0x7fcf341969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.331+0000 7fcf38f52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf34101930 0x7fcf34196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.331+0000 7fcf38f52640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf341022f0 0x7fcf3419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf38f52640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3410c3c0 con 0x7fcf341022f0 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf38f52640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3410c240 con 0x7fcf34100740 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf38f52640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3410c540 con 0x7fcf34101930 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf32d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf341022f0 0x7fcf3419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf34100740 0x7fcf341969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf32d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf341022f0 0x7fcf3419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42450/0 (socket says 172.21.15.92:42450) 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf29d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf34101930 0x7fcf34196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf32d9d640 1 -- 172.21.15.92:0/57199952 learned_addr learned my addr 172.21.15.92:0/57199952 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf32d9d640 1 -- 172.21.15.92:0/57199952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf34101930 msgr2=0x7fcf34196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf34101930 0x7fcf34196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf32d9d640 1 -- 172.21.15.92:0/57199952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf34100740 msgr2=0x7fcf341969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf34100740 0x7fcf341969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf32d9d640 1 -- 172.21.15.92:0/57199952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3419e5c0 con 0x7fcf341022f0 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf3259c640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf34100740 0x7fcf341969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf341022f0 0x7fcf3419df50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00eb10 tx=0x7fcf2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf2bfff640 1 -- 172.21.15.92:0/57199952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c004b20 con 0x7fcf341022f0 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf38f52640 1 -- 172.21.15.92:0/57199952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf34068f00 con 0x7fcf341022f0 2024-03-19T01:58:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf2bfff640 1 -- 172.21.15.92:0/57199952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c004cc0 con 0x7fcf341022f0 2024-03-19T01:58:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf2bfff640 1 -- 172.21.15.92:0/57199952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c010760 con 0x7fcf341022f0 2024-03-19T01:58:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.332+0000 7fcf38f52640 1 -- 172.21.15.92:0/57199952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf34069470 con 0x7fcf341022f0 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.334+0000 7fcf2bfff640 1 -- 172.21.15.92:0/57199952 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fcf2c005020 con 0x7fcf341022f0 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.335+0000 7fcf2bfff640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf200a28a0 0x7fcf200a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.335+0000 7fcf3259c640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf200a28a0 0x7fcf200a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.335+0000 7fcf2bfff640 1 -- 172.21.15.92:0/57199952 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcf2c0cf130 con 0x7fcf341022f0 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.335+0000 7fcf3259c640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf200a28a0 0x7fcf200a4d60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf34101790 tx=0x7fcf24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.335+0000 7fcf3bc10740 1 Processor -- start 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3bc10740 1 -- start start 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5100 0x55d8128f5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 0x55d8128fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 0x55d8128fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d812859be0 con 0x55d8128f5100 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d812802960 con 0x55d8128fac60 2024-03-19T01:58:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d812855c80 con 0x55d8128f5af0 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5100 0x55d8128f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5100 0x55d8128f5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42452/0 (socket says 172.21.15.92:42452) 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3259c640 1 -- 172.21.15.92:0/4275454447 learned_addr learned my addr 172.21.15.92:0/4275454447 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 0x55d8128fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf29d9b640 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 0x55d8128fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3259c640 1 -- 172.21.15.92:0/4275454447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 msgr2=0x55d8128fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3259c640 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 0x55d8128fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3259c640 1 -- 172.21.15.92:0/4275454447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 msgr2=0x55d8128fd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3259c640 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 0x55d8128fd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3259c640 1 -- 172.21.15.92:0/4275454447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8127f9d20 con 0x55d8128f5100 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 0x55d8128fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.336+0000 7fcf3259c640 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5100 0x55d8128f5520 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009ba0 tx=0x7fcf24014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285e49096aa1a5f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf28d99640 1 -- 172.21.15.92:0/4275454447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400f480 con 0x55d8128f5100 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf28d99640 1 -- 172.21.15.92:0/4275454447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2400fb00 con 0x55d8128f5100 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf28d99640 1 -- 172.21.15.92:0/4275454447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2401cbe0 con 0x55d8128f5100 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf3bc10740 1 -- 172.21.15.92:0/4275454447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5100 msgr2=0x55d8128f5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5100 0x55d8128f5520 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009ba0 tx=0x7fcf24014000 comp rx=0 tx=0).stop 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf3bc10740 1 -- 172.21.15.92:0/4275454447 shutdown_connections 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 0x55d8128fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 0x55d8128fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/4275454447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5100 0x55d8128f5520 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf3bc10740 1 -- 172.21.15.92:0/4275454447 >> 172.21.15.92:0/4275454447 conn(0x55d8128f0670 msgr2=0x55d8128f0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf3bc10740 1 -- 172.21.15.92:0/4275454447 shutdown_connections 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.337+0000 7fcf3bc10740 1 -- 172.21.15.92:0/4275454447 wait complete. 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3bc10740 1 -- start start 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3bc10740 1 Processor -- start 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 0x55d8128ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5af0 0x55d8128ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 0x55d812906620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d8128fdc10 con 0x55d8128f5af0 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d812815db0 con 0x55d8128fac60 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d8128fdd90 con 0x55d8128f5100 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3bc10740 10 client.0 _ll_register_callbacks cb 0x55d81281ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf29d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5af0 0x55d8128ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 0x55d8128ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf29d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5af0 0x55d8128ff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42454/0 (socket says 172.21.15.92:42454) 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf29d9b640 1 -- 172.21.15.92:0/1955885241 learned_addr learned my addr 172.21.15.92:0/1955885241 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 0x55d812906620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf29d9b640 1 -- 172.21.15.92:0/1955885241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 msgr2=0x55d8128ff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf29d9b640 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 0x55d8128ff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf29d9b640 1 -- 172.21.15.92:0/1955885241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 msgr2=0x55d812906620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf29d9b640 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 0x55d812906620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.338+0000 7fcf29d9b640 1 -- 172.21.15.92:0/1955885241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d812814c60 con 0x55d8128f5af0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf29d9b640 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5af0 0x55d8128ff5e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009900 tx=0x7fcf1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1436f86df905353 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1955885241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c005d50 con 0x55d8128f5af0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1955885241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcf1c005230 con 0x55d8128f5af0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1955885241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2400fb00 con 0x55d8128f5af0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1955885241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf1c005470 con 0x55d8128f5af0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf3bc10740 10 client.15762 fetch_fsmap learned FSMap version 29 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1955885241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d81256e210 con 0x55d8128f5af0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1955885241 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2400fb00 con 0x55d8128f5af0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1955885241 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcf1c04c030 con 0x55d8128f5af0 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf3bc10740 10 client.15762 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf3bc10740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:58:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf3bc10740 20 client.15762 populate_metadata read hostname 'smithi092' 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d81290afc0 0x55d81290d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf3bc10740 4 client.15762 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.339+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1955885241 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d812867120 con 0x55d81290afc0 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.340+0000 7fcf3259c640 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d81290afc0 0x55d81290d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.340+0000 7fcf19ffb640 10 client.15762 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.340+0000 7fcf3259c640 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d81290afc0 0x55d81290d3d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f1ca97b09115a6 server_cookie=b55ed56d74a38162 in_seq=0 out_seq=0 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1955885241 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d812867120 con 0x55d81290afc0 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf19ffb640 10 client.15762 handle_command_reply: tid=0 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 client.15762 shutdown 2024-03-19T01:58:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1955885241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5af0 msgr2=0x55d8128ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5af0 0x55d8128ff5e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009900 tx=0x7fcf1c004270 comp rx=0 tx=0).stop 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1955885241 shutdown_connections 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d81290afc0 0x55d81290d3d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128fac60 0x55d812906620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128f5af0 0x55d8128ff5e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1955885241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 0x55d8128ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf19ffb640 0 client.15762 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1955885241 >> 172.21.15.92:0/1955885241 conn(0x55d8128f0670 msgr2=0x55d8128f18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1955885241 shutdown_connections 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1955885241 wait complete. 2024-03-19T01:58:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.341+0000 7fcf3bc10740 20 client.15762 trim_cache size 0 max 16384 2024-03-19T01:58:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.355+0000 7fcf3bc10740 1 Processor -- start 2024-03-19T01:58:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.355+0000 7fcf3bc10740 1 -- start start 2024-03-19T01:58:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.355+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 0x55d8128ef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.355+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5af0 0x55d8129065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.355+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d8128ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.355+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341a6a20 con 0x55d8128fac60 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.355+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341a8560 con 0x55d8128f5af0 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.355+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341a70f0 con 0x55d8128f5100 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf32d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d8128ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf32d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d8128ef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42456/0 (socket says 172.21.15.92:42456) 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf32d9d640 1 -- 172.21.15.92:0/2259102432 learned_addr learned my addr 172.21.15.92:0/2259102432 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf3259c640 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 0x55d8128ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf29d9b640 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5af0 0x55d8129065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf32d9d640 1 -- 172.21.15.92:0/2259102432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 msgr2=0x55d8128ef220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 0x55d8128ef220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf32d9d640 1 -- 172.21.15.92:0/2259102432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5af0 msgr2=0x55d8129065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5af0 0x55d8129065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf32d9d640 1 -- 172.21.15.92:0/2259102432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81263a620 con 0x55d8128fac60 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf3259c640 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 0x55d8128ef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d8128ef760 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0ccc90 tx=0x7fcf2c09b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccfba85207fa0e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf1bfff640 1 -- 172.21.15.92:0/2259102432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c09fc30 con 0x55d8128fac60 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf1bfff640 1 -- 172.21.15.92:0/2259102432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c0b3070 con 0x55d8128fac60 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.356+0000 7fcf1bfff640 1 -- 172.21.15.92:0/2259102432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c005600 con 0x55d8128fac60 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 -- 172.21.15.92:0/2259102432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 msgr2=0x55d8128ef760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d8128ef760 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0ccc90 tx=0x7fcf2c09b720 comp rx=0 tx=0).stop 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 -- 172.21.15.92:0/2259102432 shutdown_connections 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d8128ef760 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5af0 0x55d8129065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/2259102432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5100 0x55d8128ef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 -- 172.21.15.92:0/2259102432 >> 172.21.15.92:0/2259102432 conn(0x55d8128f0670 msgr2=0x55d8128ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 -- 172.21.15.92:0/2259102432 shutdown_connections 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 -- 172.21.15.92:0/2259102432 wait complete. 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 -- start start 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.357+0000 7fcf3bc10740 1 Processor -- start 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5100 0x55d812903cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 0x55d8129041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3bc10740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d812905200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341a7e90 con 0x55d8128fac60 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341a77c0 con 0x55d8128f5100 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3bc10740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341a4a00 con 0x55d8128f5af0 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3bc10740 10 client.0 _ll_register_callbacks cb 0x7fcf1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf32d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d812905200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5100 0x55d812903cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf32d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d812905200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42472/0 (socket says 172.21.15.92:42472) 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf32d9d640 1 -- 172.21.15.92:0/1862541055 learned_addr learned my addr 172.21.15.92:0/1862541055 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf29d9b640 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 0x55d8129041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf32d9d640 1 -- 172.21.15.92:0/1862541055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 msgr2=0x55d8129041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 0x55d8129041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf32d9d640 1 -- 172.21.15.92:0/1862541055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5100 msgr2=0x55d812903cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5100 0x55d812903cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf32d9d640 1 -- 172.21.15.92:0/1862541055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8129057f0 con 0x55d8128fac60 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3259c640 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5100 0x55d812903cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf32d9d640 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d812905200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c09ba70 tx=0x7fcf2c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca086080afd8763 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1862541055 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0a4280 con 0x55d8128fac60 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1862541055 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcf2c0b4ba0 con 0x55d8128fac60 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1862541055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2c0b3070 con 0x55d8128fac60 2024-03-19T01:58:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1862541055 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf2c09fc30 con 0x55d8128fac60 2024-03-19T01:58:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.358+0000 7fcf3bc10740 10 client.15768 fetch_fsmap learned FSMap version 29 2024-03-19T01:58:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1862541055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d812971310 con 0x55d8128fac60 2024-03-19T01:58:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1862541055 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c0b3070 con 0x55d8128fac60 2024-03-19T01:58:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1862541055 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcf2c09ab40 con 0x55d8128fac60 2024-03-19T01:58:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf3bc10740 10 client.15768 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:58:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf3bc10740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:58:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf3bc10740 20 client.15768 populate_metadata read hostname 'smithi092' 2024-03-19T01:58:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d812971c80 0x55d8129740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf3bc10740 4 client.15768 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:58:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1862541055 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8128ebf50 con 0x55d812971c80 2024-03-19T01:58:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf3259c640 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d812971c80 0x55d8129740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf19ffb640 10 client.15768 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.359+0000 7fcf3259c640 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d812971c80 0x55d8129740a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2763c135eeb52750 server_cookie=61f911c8c61a6a32 in_seq=0 out_seq=0 2024-03-19T01:58:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf19ffb640 1 -- 172.21.15.92:0/1862541055 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8128ebf50 con 0x55d812971c80 2024-03-19T01:58:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf19ffb640 10 client.15768 handle_command_reply: tid=0 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 client.15768 shutdown 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1862541055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 msgr2=0x55d812905200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d812905200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c09ba70 tx=0x7fcf2c00c990 comp rx=0 tx=0).stop 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1862541055 shutdown_connections 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d812971c80 0x55d8129740a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8128fac60 0x55d812905200 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8128f5af0 0x55d8129041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf19ffb640 0 client.15768 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 --2- 172.21.15.92:0/1862541055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8128f5100 0x55d812903cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1862541055 >> 172.21.15.92:0/1862541055 conn(0x55d8128f0670 msgr2=0x55d8128ef220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1862541055 shutdown_connections 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 1 -- 172.21.15.92:0/1862541055 wait complete. 2024-03-19T01:58:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.366+0000 7fcf3bc10740 20 client.15768 trim_cache size 0 max 16384 2024-03-19T01:58:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:58:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:58:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:58:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 -- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf200a28a0 msgr2=0x7fcf200a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf200a28a0 0x7fcf200a4d60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf34101790 tx=0x7fcf24005f90 comp rx=0 tx=0).stop 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 -- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf341022f0 msgr2=0x7fcf3419df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf341022f0 0x7fcf3419df50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00eb10 tx=0x7fcf2c00c2d0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 -- 172.21.15.92:0/57199952 shutdown_connections 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf341022f0 0x7fcf3419df50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf200a28a0 0x7fcf200a4d60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf34101930 0x7fcf34196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 --2- 172.21.15.92:0/57199952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf34100740 0x7fcf341969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 -- 172.21.15.92:0/57199952 >> 172.21.15.92:0/57199952 conn(0x7fcf340fbcb0 msgr2=0x7fcf34104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 -- 172.21.15.92:0/57199952 shutdown_connections 2024-03-19T01:58:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:23.367+0000 7fcf38f52640 1 -- 172.21.15.92:0/57199952 wait complete. 2024-03-19T01:58:24.898 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:24 smithi144 ceph-mon[67199]: pgmap v519: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:58:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:24 smithi159 ceph-mon[67110]: pgmap v519: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:58:25.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:24 smithi092 ceph-mon[62574]: pgmap v519: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:58:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:26 smithi159 ceph-mon[67110]: pgmap v520: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T01:58:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:26 smithi092 ceph-mon[62574]: pgmap v520: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T01:58:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:26 smithi144 ceph-mon[67199]: pgmap v520: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T01:58:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:28 smithi159 ceph-mon[67110]: pgmap v521: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:28 smithi092 ceph-mon[62574]: pgmap v521: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:28 smithi144 ceph-mon[67199]: pgmap v521: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:30 smithi159 ceph-mon[67110]: pgmap v522: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:30 smithi092 ceph-mon[62574]: pgmap v522: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:30 smithi144 ceph-mon[67199]: pgmap v522: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:32 smithi159 ceph-mon[67110]: pgmap v523: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:58:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:32 smithi092 ceph-mon[62574]: pgmap v523: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:58:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:32 smithi144 ceph-mon[67199]: pgmap v523: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:58:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:34 smithi159 ceph-mon[67110]: pgmap v524: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:58:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:34 smithi092 ceph-mon[62574]: pgmap v524: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:58:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:34 smithi144 ceph-mon[67199]: pgmap v524: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:58:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.457+0000 7f7bdb236640 1 -- 172.21.15.92:0/1516688397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4101950 msgr2=0x7f7bd4101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.457+0000 7f7bdb236640 1 --2- 172.21.15.92:0/1516688397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4101950 0x7f7bd4101dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009900 tx=0x7f7bc402bc30 comp rx=0 tx=0).stop 2024-03-19T01:58:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.457+0000 7f7bdb236640 1 -- 172.21.15.92:0/1516688397 shutdown_connections 2024-03-19T01:58:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.457+0000 7f7bdb236640 1 --2- 172.21.15.92:0/1516688397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bd4102310 0x7f7bd4108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.457+0000 7f7bdb236640 1 --2- 172.21.15.92:0/1516688397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4101950 0x7f7bd4101dd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.457+0000 7f7bdb236640 1 --2- 172.21.15.92:0/1516688397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd4100760 0x7f7bd4100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.457+0000 7f7bdb236640 1 -- 172.21.15.92:0/1516688397 >> 172.21.15.92:0/1516688397 conn(0x7f7bd40fbcd0 msgr2=0x7f7bd40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.457+0000 7f7bdb236640 1 -- 172.21.15.92:0/1516688397 shutdown_connections 2024-03-19T01:58:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.457+0000 7f7bdb236640 1 -- 172.21.15.92:0/1516688397 wait complete. 2024-03-19T01:58:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bdb236640 1 Processor -- start 2024-03-19T01:58:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bdb236640 1 -- start start 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bdb236640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd4100760 0x7f7bd406ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bdb236640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bd4101950 0x7f7bd419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bdb236640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4102310 0x7f7bd419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bdb236640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd410c3c0 con 0x7f7bd4102310 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bdb236640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd410c240 con 0x7f7bd4101950 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bdb236640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd410c540 con 0x7f7bd4100760 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bd97d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4102310 0x7f7bd419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bd97d3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4102310 0x7f7bd419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58916/0 (socket says 172.21.15.92:58916) 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bd97d3640 1 -- 172.21.15.92:0/3199484843 learned_addr learned my addr 172.21.15.92:0/3199484843 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd4100760 0x7f7bd406ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.458+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bd4101950 0x7f7bd419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bd97d3640 1 -- 172.21.15.92:0/3199484843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd4100760 msgr2=0x7f7bd406ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bd97d3640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd4100760 0x7f7bd406ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bd97d3640 1 -- 172.21.15.92:0/3199484843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bd4101950 msgr2=0x7f7bd419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bd97d3640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bd4101950 0x7f7bd419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bd97d3640 1 -- 172.21.15.92:0/3199484843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd419f750 con 0x7f7bd4102310 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bd97d3640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4102310 0x7f7bd419ee40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00eb10 tx=0x7f7bcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bc9ffb640 1 -- 172.21.15.92:0/3199484843 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc004b20 con 0x7f7bd4102310 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bc9ffb640 1 -- 172.21.15.92:0/3199484843 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bcc004cc0 con 0x7f7bd4102310 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bdb236640 1 -- 172.21.15.92:0/3199484843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd419f9e0 con 0x7f7bd4102310 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bc9ffb640 1 -- 172.21.15.92:0/3199484843 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc010760 con 0x7f7bd4102310 2024-03-19T01:58:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.459+0000 7f7bdb236640 1 -- 172.21.15.92:0/3199484843 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd419feb0 con 0x7f7bd4102310 2024-03-19T01:58:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bdde29740 1 Processor -- start 2024-03-19T01:58:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bc9ffb640 1 -- 172.21.15.92:0/3199484843 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f7bcc005020 con 0x7f7bd4102310 2024-03-19T01:58:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bdde29740 1 -- start start 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1ad100 0x557cbd1ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 0x557cbd1b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bdde29740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557cbd111be0 con 0x557cbd1b2c60 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bdde29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557cbd0ba960 con 0x557cbd1adaf0 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bdde29740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557cbd10dc80 con 0x557cbd1ad100 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1ad100 0x557cbd1ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bd8fd2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 0x557cbd1b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1ad100 0x557cbd1ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:39166/0 (socket says 172.21.15.92:39166) 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.462+0000 7f7bcbfff640 1 -- 172.21.15.92:0/3957099282 learned_addr learned my addr 172.21.15.92:0/3957099282 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bc9ffb640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7bbc0a28a0 0x7f7bbc0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bd97d3640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7bbc0a28a0 0x7f7bbc0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bd8fd2640 1 -- 172.21.15.92:0/3957099282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1ad100 msgr2=0x557cbd1ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/3957099282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1ad100 0x557cbd1ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bd8fd2640 1 -- 172.21.15.92:0/3957099282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 msgr2=0x557cbd1b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/3957099282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bc9ffb640 1 -- 172.21.15.92:0/3199484843 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7bcc0cefc0 con 0x7f7bd4102310 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bd8fd2640 1 -- 172.21.15.92:0/3957099282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cbd0b1d20 con 0x557cbd1adaf0 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/3957099282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1ad100 0x557cbd1ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/3957099282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 0x557cbd1b25f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009bf0 tx=0x7f7bd0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cb5c0b94e290139 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bd97d3640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7bbc0a28a0 0x7f7bbc0a4d60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc098340 tx=0x7f7bcc090e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bbaffd640 1 -- 172.21.15.92:0/3957099282 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0009da0 con 0x557cbd1adaf0 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bbaffd640 1 -- 172.21.15.92:0/3957099282 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd0005020 con 0x557cbd1adaf0 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.463+0000 7f7bbaffd640 1 -- 172.21.15.92:0/3957099282 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0004a70 con 0x557cbd1adaf0 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.464+0000 7f7bdde29740 1 -- 172.21.15.92:0/3957099282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 msgr2=0x557cbd1b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.464+0000 7f7bdde29740 1 --2- 172.21.15.92:0/3957099282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 0x557cbd1b25f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009bf0 tx=0x7f7bd0011810 comp rx=0 tx=0).stop 2024-03-19T01:58:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.464+0000 7f7bdde29740 1 -- 172.21.15.92:0/3957099282 shutdown_connections 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.464+0000 7f7bdde29740 1 --2- 172.21.15.92:0/3957099282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.464+0000 7f7bdde29740 1 --2- 172.21.15.92:0/3957099282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 0x557cbd1b25f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.464+0000 7f7bdde29740 1 --2- 172.21.15.92:0/3957099282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1ad100 0x557cbd1ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.464+0000 7f7bdde29740 1 -- 172.21.15.92:0/3957099282 >> 172.21.15.92:0/3957099282 conn(0x557cbd1a8670 msgr2=0x557cbd1a8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.464+0000 7f7bdde29740 1 -- 172.21.15.92:0/3957099282 shutdown_connections 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.464+0000 7f7bdde29740 1 -- 172.21.15.92:0/3957099282 wait complete. 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bdde29740 1 -- start start 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bdde29740 1 Processor -- start 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 0x557cbd1b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 0x557cbd1b7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1be660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bdde29740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557cbd1b5c10 con 0x557cbd1b2c60 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bdde29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557cbd0cddb0 con 0x557cbd1ad100 2024-03-19T01:58:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bdde29740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557cbd1b5d90 con 0x557cbd1adaf0 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bdde29740 10 client.0 _ll_register_callbacks cb 0x557cbd0d2e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1be660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58940/0 (socket says 172.21.15.92:58940) 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bcbfff640 1 -- 172.21.15.92:0/1431359448 learned_addr learned my addr 172.21.15.92:0/1431359448 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 0x557cbd1b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.465+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 0x557cbd1b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bcbfff640 1 -- 172.21.15.92:0/1431359448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 msgr2=0x557cbd1b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 0x557cbd1b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bcbfff640 1 -- 172.21.15.92:0/1431359448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 msgr2=0x557cbd1b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 0x557cbd1b70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bcbfff640 1 -- 172.21.15.92:0/1431359448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cbd0ccc60 con 0x557cbd1b2c60 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 0x557cbd1b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1be660 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7bc402cca0 tx=0x7f7bc402c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3aca501e85ad022 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:58:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bb3fff640 1 -- 172.21.15.92:0/1431359448 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc402cd00 con 0x557cbd1b2c60 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bb3fff640 1 -- 172.21.15.92:0/1431359448 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7bc4003c10 con 0x557cbd1b2c60 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bdde29740 1 -- 172.21.15.92:0/1431359448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bd0005020 con 0x557cbd1b2c60 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bb3fff640 1 -- 172.21.15.92:0/1431359448 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7bc4003e10 con 0x557cbd1b2c60 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bdde29740 10 client.15780 fetch_fsmap learned FSMap version 29 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bdde29740 1 -- 172.21.15.92:0/1431359448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557cbce26210 con 0x557cbd1b2c60 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bb3fff640 1 -- 172.21.15.92:0/1431359448 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd0005020 con 0x557cbd1b2c60 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bb3fff640 1 -- 172.21.15.92:0/1431359448 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7bc404d420 con 0x557cbd1b2c60 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bdde29740 10 client.15780 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bdde29740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bdde29740 20 client.15780 populate_metadata read hostname 'smithi092' 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bdde29740 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557cbd1c2f90 0x557cbd1c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bdde29740 4 client.15780 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.466+0000 7f7bdde29740 1 -- 172.21.15.92:0/1431359448 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cbd11f120 con 0x557cbd1c2f90 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.467+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557cbd1c2f90 0x557cbd1c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.467+0000 7f7bb3fff640 10 client.15780 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.467+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557cbd1c2f90 0x557cbd1c53c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5fb3f6aec1781e9 server_cookie=638dccfe56e4a83 in_seq=0 out_seq=0 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bb3fff640 1 -- 172.21.15.92:0/1431359448 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557cbd11f120 con 0x557cbd1c2f90 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bb3fff640 10 client.15780 handle_command_reply: tid=0 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 client.15780 shutdown 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 -- 172.21.15.92:0/1431359448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 msgr2=0x557cbd1be660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1be660 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7bc402cca0 tx=0x7f7bc402c6b0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 -- 172.21.15.92:0/1431359448 shutdown_connections 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557cbd1c2f90 0x557cbd1c53c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1be660 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bb3fff640 0 client.15780 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 0x557cbd1b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 --2- 172.21.15.92:0/1431359448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 0x557cbd1b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 -- 172.21.15.92:0/1431359448 >> 172.21.15.92:0/1431359448 conn(0x557cbd1a8670 msgr2=0x557cbd1a98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 -- 172.21.15.92:0/1431359448 shutdown_connections 2024-03-19T01:58:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 1 -- 172.21.15.92:0/1431359448 wait complete. 2024-03-19T01:58:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.468+0000 7f7bdde29740 20 client.15780 trim_cache size 0 max 16384 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bdde29740 1 Processor -- start 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bdde29740 1 -- start start 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 0x557cbd1ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 0x557cbd1be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bdde29740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd41a8b50 con 0x557cbd1b2c60 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bdde29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd41aa690 con 0x557cbd1ad100 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bdde29740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd41a9220 con 0x557cbd1adaf0 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58956/0 (socket says 172.21.15.92:58956) 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bcbfff640 1 -- 172.21.15.92:0/3560880209 learned_addr learned my addr 172.21.15.92:0/3560880209 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bd8fd2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 0x557cbd1be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/3560880209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 0x557cbd1ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bcbfff640 1 -- 172.21.15.92:0/3560880209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 msgr2=0x557cbd1be620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/3560880209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 0x557cbd1be620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bcbfff640 1 -- 172.21.15.92:0/3560880209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 msgr2=0x557cbd1ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/3560880209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 0x557cbd1ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.483+0000 7f7bcbfff640 1 -- 172.21.15.92:0/3560880209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cbd1a3ee0 con 0x557cbd1b2c60 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/3560880209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1aead0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40096c0 tx=0x7f7bc4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192a3157dd767fb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bba7fc640 1 -- 172.21.15.92:0/3560880209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4003a10 con 0x557cbd1b2c60 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bba7fc640 1 -- 172.21.15.92:0/3560880209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc4003bb0 con 0x557cbd1b2c60 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bba7fc640 1 -- 172.21.15.92:0/3560880209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4034900 con 0x557cbd1b2c60 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bdde29740 1 -- 172.21.15.92:0/3560880209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 msgr2=0x557cbd1aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bdde29740 1 --2- 172.21.15.92:0/3560880209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1aead0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40096c0 tx=0x7f7bc4005fd0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bdde29740 1 -- 172.21.15.92:0/3560880209 shutdown_connections 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bdde29740 1 --2- 172.21.15.92:0/3560880209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1b2c60 0x557cbd1aead0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bdde29740 1 --2- 172.21.15.92:0/3560880209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1adaf0 0x557cbd1be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bdde29740 1 --2- 172.21.15.92:0/3560880209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1ad100 0x557cbd1ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bdde29740 1 -- 172.21.15.92:0/3560880209 >> 172.21.15.92:0/3560880209 conn(0x557cbd1a8670 msgr2=0x557cbd1a7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bdde29740 1 -- 172.21.15.92:0/3560880209 shutdown_connections 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.484+0000 7f7bdde29740 1 -- 172.21.15.92:0/3560880209 wait complete. 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bdde29740 1 -- start start 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bdde29740 1 Processor -- start 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1ad100 0x557cbd1bc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 0x557cbd1bc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bdde29740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1b2c60 0x557cbd1bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bdde29740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd41a9fc0 con 0x557cbd1ad100 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bdde29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd41a98f0 con 0x557cbd1adaf0 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bdde29740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd41a6b30 con 0x557cbd1b2c60 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bdde29740 10 client.0 _ll_register_callbacks cb 0x7f7bc4005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:58:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1ad100 0x557cbd1bc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1ad100 0x557cbd1bc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58960/0 (socket says 172.21.15.92:58960) 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bcbfff640 1 -- 172.21.15.92:0/2856421705 learned_addr learned my addr 172.21.15.92:0/2856421705 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 0x557cbd1bc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1b2c60 0x557cbd1bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bcbfff640 1 -- 172.21.15.92:0/2856421705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1b2c60 msgr2=0x557cbd1bd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1b2c60 0x557cbd1bd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bcbfff640 1 -- 172.21.15.92:0/2856421705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 msgr2=0x557cbd1bc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.485+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 0x557cbd1bc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bcbfff640 1 -- 172.21.15.92:0/2856421705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cbcef2620 con 0x557cbd1ad100 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bcbfff640 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1ad100 0x557cbd1bc280 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7bc402c8e0 tx=0x7f7bc4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f54fd8d4c62e7d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bb3fff640 1 -- 172.21.15.92:0/2856421705 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc404d650 con 0x557cbd1ad100 2024-03-19T01:58:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bb3fff640 1 -- 172.21.15.92:0/2856421705 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f7bc4041430 con 0x557cbd1ad100 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bdde29740 1 -- 172.21.15.92:0/2856421705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc4003bb0 con 0x557cbd1ad100 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bb3fff640 1 -- 172.21.15.92:0/2856421705 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7bc4034680 con 0x557cbd1ad100 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bdde29740 10 client.15786 fetch_fsmap learned FSMap version 29 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bdde29740 1 -- 172.21.15.92:0/2856421705 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557cbd2296b0 con 0x557cbd1ad100 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bb3fff640 1 -- 172.21.15.92:0/2856421705 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc4003bb0 con 0x557cbd1ad100 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bb3fff640 1 -- 172.21.15.92:0/2856421705 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7bc4041a10 con 0x557cbd1ad100 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bdde29740 10 client.15786 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bdde29740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bdde29740 20 client.15786 populate_metadata read hostname 'smithi092' 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bdde29740 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557cbd229f50 0x557cbd22c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bdde29740 4 client.15786 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.486+0000 7f7bdde29740 1 -- 172.21.15.92:0/2856421705 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557cbd119470 con 0x557cbd229f50 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.487+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557cbd229f50 0x557cbd22c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.487+0000 7f7bb3fff640 10 client.15786 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.487+0000 7f7bd8fd2640 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557cbd229f50 0x557cbd22c370 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1282ea625929dd server_cookie=57f27f9c036717e7 in_seq=0 out_seq=0 2024-03-19T01:58:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bb3fff640 1 -- 172.21.15.92:0/2856421705 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557cbd119470 con 0x557cbd229f50 2024-03-19T01:58:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bb3fff640 10 client.15786 handle_command_reply: tid=0 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 client.15786 shutdown 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 -- 172.21.15.92:0/2856421705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1ad100 msgr2=0x557cbd1bc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1ad100 0x557cbd1bc280 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7bc402c8e0 tx=0x7f7bc4005d50 comp rx=0 tx=0).stop 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 -- 172.21.15.92:0/2856421705 shutdown_connections 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557cbd229f50 0x557cbd22c370 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cbd1b2c60 0x557cbd1bd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bb3fff640 0 client.15786 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557cbd1adaf0 0x557cbd1bc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 --2- 172.21.15.92:0/2856421705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557cbd1ad100 0x557cbd1bc280 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 -- 172.21.15.92:0/2856421705 >> 172.21.15.92:0/2856421705 conn(0x557cbd1a8670 msgr2=0x557cbd1ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 -- 172.21.15.92:0/2856421705 shutdown_connections 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 1 -- 172.21.15.92:0/2856421705 wait complete. 2024-03-19T01:58:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.494+0000 7f7bdde29740 20 client.15786 trim_cache size 0 max 16384 2024-03-19T01:58:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:58:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:58:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:58:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 -- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7bbc0a28a0 msgr2=0x7f7bbc0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7bbc0a28a0 0x7f7bbc0a4d60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc098340 tx=0x7f7bcc090e50 comp rx=0 tx=0).stop 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 -- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4102310 msgr2=0x7f7bd419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4102310 0x7f7bd419ee40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00eb10 tx=0x7f7bcc00c2d0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 -- 172.21.15.92:0/3199484843 shutdown_connections 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bd4102310 0x7f7bd419ee40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bd4101950 0x7f7bd419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f7bbc0a28a0 0x7f7bbc0a4d60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 --2- 172.21.15.92:0/3199484843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd4100760 0x7f7bd406ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 -- 172.21.15.92:0/3199484843 >> 172.21.15.92:0/3199484843 conn(0x7f7bd40fbcd0 msgr2=0x7f7bd4104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 -- 172.21.15.92:0/3199484843 shutdown_connections 2024-03-19T01:58:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:36.495+0000 7f7bdb236640 1 -- 172.21.15.92:0/3199484843 wait complete. 2024-03-19T01:58:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:36 smithi159 ceph-mon[67110]: pgmap v525: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T01:58:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:36 smithi092 ceph-mon[62574]: pgmap v525: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T01:58:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:36 smithi144 ceph-mon[67199]: pgmap v525: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T01:58:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:38 smithi159 ceph-mon[67110]: pgmap v526: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T01:58:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:38 smithi092 ceph-mon[62574]: pgmap v526: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T01:58:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:38 smithi144 ceph-mon[67199]: pgmap v526: 97 pgs: 97 active+clean; 2.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T01:58:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:40 smithi159 ceph-mon[67110]: pgmap v527: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:58:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:40 smithi092 ceph-mon[62574]: pgmap v527: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:58:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:40 smithi144 ceph-mon[67199]: pgmap v527: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T01:58:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:42 smithi159 ceph-mon[67110]: pgmap v528: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:58:43.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:42 smithi092 ceph-mon[62574]: pgmap v528: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:58:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:42 smithi144 ceph-mon[67199]: pgmap v528: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T01:58:44.959 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:44 smithi159 ceph-mon[67110]: pgmap v529: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:45.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:44 smithi092 ceph-mon[62574]: pgmap v529: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:45.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:44 smithi144 ceph-mon[67199]: pgmap v529: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:46 smithi159 ceph-mon[67110]: pgmap v530: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:46 smithi092 ceph-mon[62574]: pgmap v530: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:46 smithi144 ceph-mon[67199]: pgmap v530: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:58:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:48 smithi159 ceph-mon[67110]: pgmap v531: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:58:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:48 smithi092 ceph-mon[62574]: pgmap v531: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:58:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:48 smithi144 ceph-mon[67199]: pgmap v531: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:58:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.584+0000 7f4bc908c640 1 -- 172.21.15.92:0/1499770776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4102a70 msgr2=0x7f4bc4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.584+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1499770776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4102a70 0x7f4bc4102ef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009900 tx=0x7f4bb402bc30 comp rx=0 tx=0).stop 2024-03-19T01:58:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.584+0000 7f4bc908c640 1 -- 172.21.15.92:0/1499770776 shutdown_connections 2024-03-19T01:58:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.584+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1499770776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bc4103430 0x7f4bc4109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.584+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1499770776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4102a70 0x7f4bc4102ef0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.584+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1499770776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bc4101880 0x7f4bc4101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.584+0000 7f4bc908c640 1 -- 172.21.15.92:0/1499770776 >> 172.21.15.92:0/1499770776 conn(0x7f4bc40fcdf0 msgr2=0x7f4bc40ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:49.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.584+0000 7f4bc908c640 1 -- 172.21.15.92:0/1499770776 shutdown_connections 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.584+0000 7f4bc908c640 1 -- 172.21.15.92:0/1499770776 wait complete. 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.585+0000 7f4bc908c640 1 Processor -- start 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.585+0000 7f4bc908c640 1 -- start start 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.585+0000 7f4bc908c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4101880 0x7f4bc4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.585+0000 7f4bc908c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bc4102a70 0x7f4bc4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.585+0000 7f4bc908c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bc4103430 0x7f4bc419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.585+0000 7f4bc908c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc410d500 con 0x7f4bc4101880 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.585+0000 7f4bc908c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc410d380 con 0x7f4bc4102a70 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.585+0000 7f4bc908c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc410d680 con 0x7f4bc4103430 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.585+0000 7f4bc2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4101880 0x7f4bc4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4101880 0x7f4bc4197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52190/0 (socket says 172.21.15.92:52190) 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc2d9d640 1 -- 172.21.15.92:0/1617813014 learned_addr learned my addr 172.21.15.92:0/1617813014 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc359e640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bc4103430 0x7f4bc419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc259c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bc4102a70 0x7f4bc4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc359e640 1 -- 172.21.15.92:0/1617813014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bc4102a70 msgr2=0x7f4bc4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc359e640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bc4102a70 0x7f4bc4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc359e640 1 -- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4101880 msgr2=0x7f4bc4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc359e640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4101880 0x7f4bc4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc359e640 1 -- 172.21.15.92:0/1617813014 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc419f6e0 con 0x7f4bc4103430 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4101880 0x7f4bc4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc259c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bc4102a70 0x7f4bc4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc359e640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bc4103430 0x7f4bc419f070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009fd0 tx=0x7f4bbc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bbbfff640 1 -- 172.21.15.92:0/1617813014 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00ce20 con 0x7f4bc4103430 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bbbfff640 1 -- 172.21.15.92:0/1617813014 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc005020 con 0x7f4bc4103430 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc908c640 1 -- 172.21.15.92:0/1617813014 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc419f9d0 con 0x7f4bc4103430 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bbbfff640 1 -- 172.21.15.92:0/1617813014 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc010720 con 0x7f4bc4103430 2024-03-19T01:58:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.586+0000 7f4bc908c640 1 -- 172.21.15.92:0/1617813014 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc419fef0 con 0x7f4bc4103430 2024-03-19T01:58:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.588+0000 7f4bbbfff640 1 -- 172.21.15.92:0/1617813014 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f4bbc005540 con 0x7f4bc4103430 2024-03-19T01:58:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bcbc7e740 1 Processor -- start 2024-03-19T01:58:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bbbfff640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4bb00a28a0 0x7f4bb00a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bcbc7e740 1 -- start start 2024-03-19T01:58:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bc259c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4bb00a28a0 0x7f4bb00a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bbbfff640 1 -- 172.21.15.92:0/1617813014 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4bbc0cf020 con 0x7f4bc4103430 2024-03-19T01:58:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0ed520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a2ab051be0 con 0x55a2ab0f2c60 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a2aaffa960 con 0x55a2ab0ed100 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2ab04dc80 con 0x55a2ab0edaf0 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bc359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bc359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52194/0 (socket says 172.21.15.92:52194) 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bc359e640 1 -- 172.21.15.92:0/3738248708 learned_addr learned my addr 172.21.15.92:0/3738248708 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.589+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0ed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bc259c640 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bc2d9d640 1 -- 172.21.15.92:0/3738248708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0edaf0 msgr2=0x55a2ab0f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bc2d9d640 1 -- 172.21.15.92:0/3738248708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0f2c60 msgr2=0x55a2ab0f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bc259c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4bb00a28a0 0x7f4bb00a4d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4bb402c7b0 tx=0x7f4bb40095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bc2d9d640 1 -- 172.21.15.92:0/3738248708 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2aaff1d20 con 0x55a2ab0ed100 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bc359e640 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0ed520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0114c0 tx=0x7f4bac011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6887e338c08f4b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bb97fa640 1 -- 172.21.15.92:0/3738248708 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac005020 con 0x55a2ab0ed100 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bb97fa640 1 -- 172.21.15.92:0/3738248708 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bac0047a0 con 0x55a2ab0ed100 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.590+0000 7f4bb97fa640 1 -- 172.21.15.92:0/3738248708 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00f8f0 con 0x55a2ab0ed100 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3738248708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 msgr2=0x55a2ab0ed520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0ed520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bac0114c0 tx=0x7f4bac011980 comp rx=0 tx=0).stop 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3738248708 shutdown_connections 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/3738248708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0ed520 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3738248708 >> 172.21.15.92:0/3738248708 conn(0x55a2ab0e8670 msgr2=0x55a2ab0e8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3738248708 shutdown_connections 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3738248708 wait complete. 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 -- start start 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 Processor -- start 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.591+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0ed100 0x55a2ab0f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0fe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a2ab0f5c10 con 0x55a2ab0edaf0 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a2ab00ddb0 con 0x55a2ab0f2c60 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2ab0f5d90 con 0x55a2ab0ed100 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bcbc7e740 10 client.0 _ll_register_callbacks cb 0x55a2ab012e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:58:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc2d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0ed100 0x55a2ab0f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52208/0 (socket says 172.21.15.92:52208) 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc259c640 1 -- 172.21.15.92:0/2311743329 learned_addr learned my addr 172.21.15.92:0/2311743329 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc2d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0ed100 0x55a2ab0f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:44878/0 (socket says 172.21.15.92:44878) 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc359e640 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0fe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc259c640 1 -- 172.21.15.92:0/2311743329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0ed100 msgr2=0x55a2ab0f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc259c640 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0ed100 0x55a2ab0f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc259c640 1 -- 172.21.15.92:0/2311743329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0f2c60 msgr2=0x55a2ab0fe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc259c640 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0fe600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc259c640 1 -- 172.21.15.92:0/2311743329 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2ab00cc60 con 0x55a2ab0edaf0 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bc259c640 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f75c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009900 tx=0x7f4bb402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f4865e2f98849 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:49.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4baa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2311743329 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4005630 con 0x55a2ab0edaf0 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2311743329 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4bb40046c0 con 0x55a2ab0edaf0 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2311743329 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bac0047a0 con 0x55a2ab0edaf0 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2311743329 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bb40048c0 con 0x55a2ab0edaf0 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bcbc7e740 10 client.15798 fetch_fsmap learned FSMap version 29 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.592+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2311743329 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2aad66210 con 0x55a2ab0edaf0 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2311743329 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bac0047a0 con 0x55a2ab0edaf0 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2311743329 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4bb4034440 con 0x55a2ab0edaf0 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4bcbc7e740 10 client.15798 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4bcbc7e740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4bcbc7e740 20 client.15798 populate_metadata read hostname 'smithi092' 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55a2ab102f20 0x55a2ab105350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4bcbc7e740 4 client.15798 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2311743329 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2ab05f120 con 0x55a2ab102f20 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55a2ab102f20 0x55a2ab105350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55a2ab102f20 0x55a2ab105350 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfef16d0d3cc5d7b server_cookie=76b268171ad79da5 in_seq=0 out_seq=0 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.593+0000 7f4baa7fc640 10 client.15798 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2311743329 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a2ab05f120 con 0x55a2ab102f20 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4baa7fc640 10 client.15798 handle_command_reply: tid=0 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 client.15798 shutdown 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2311743329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 msgr2=0x55a2ab0f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f75c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009900 tx=0x7f4bb402bc30 comp rx=0 tx=0).stop 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2311743329 shutdown_connections 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55a2ab102f20 0x55a2ab105350 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4baa7fc640 0 client.15798 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0f2c60 0x55a2ab0fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0f75c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2311743329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0ed100 0x55a2ab0f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2311743329 >> 172.21.15.92:0/2311743329 conn(0x55a2ab0e8670 msgr2=0x55a2ab0e98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2311743329 shutdown_connections 2024-03-19T01:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2311743329 wait complete. 2024-03-19T01:58:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.594+0000 7f4bcbc7e740 20 client.15798 trim_cache size 0 max 16384 2024-03-19T01:58:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.613+0000 7f4bcbc7e740 1 Processor -- start 2024-03-19T01:58:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.613+0000 7f4bcbc7e740 1 -- start start 2024-03-19T01:58:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.613+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 0x55a2ab109c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41a8c30 con 0x55a2ab0edaf0 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41aa770 con 0x55a2ab0ed100 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41a9300 con 0x55a2ab0f2c60 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0ebfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52216/0 (socket says 172.21.15.92:52216) 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc259c640 1 -- 172.21.15.92:0/3451401080 learned_addr learned my addr 172.21.15.92:0/3451401080 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc359e640 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 0x55a2ab109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc259c640 1 -- 172.21.15.92:0/3451401080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 msgr2=0x55a2ab109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc259c640 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 0x55a2ab109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc259c640 1 -- 172.21.15.92:0/3451401080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 msgr2=0x55a2ab0e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc259c640 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc259c640 1 -- 172.21.15.92:0/3451401080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2aae32620 con 0x55a2ab0edaf0 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc359e640 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 0x55a2ab109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bc259c640 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0ebfe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4bb402c170 tx=0x7f4bb4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175c95c9a169c25c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bb8ff9640 1 -- 172.21.15.92:0/3451401080 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4034440 con 0x55a2ab0edaf0 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bb8ff9640 1 -- 172.21.15.92:0/3451401080 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4059030 con 0x55a2ab0edaf0 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.614+0000 7f4bb8ff9640 1 -- 172.21.15.92:0/3451401080 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4004d50 con 0x55a2ab0edaf0 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3451401080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 msgr2=0x55a2ab0ebfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0ebfe0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4bb402c170 tx=0x7f4bb4005bd0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3451401080 shutdown_connections 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 0x55a2ab109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0edaf0 0x55a2ab0ebfe0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/3451401080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0ed100 0x55a2ab0e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3451401080 >> 172.21.15.92:0/3451401080 conn(0x55a2ab0e8670 msgr2=0x55a2ab0eb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3451401080 shutdown_connections 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/3451401080 wait complete. 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 -- start start 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.615+0000 7f4bcbc7e740 1 Processor -- start 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0ed100 0x55a2ab165180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0edaf0 0x55a2ab1656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 0x55a2ab16c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41aa0a0 con 0x55a2ab0ed100 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41a99d0 con 0x55a2ab0edaf0 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc41a6c10 con 0x55a2ab0f2c60 2024-03-19T01:58:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 10 client.0 _ll_register_callbacks cb 0x7f4bbc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0ed100 0x55a2ab165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0ed100 0x55a2ab165180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52232/0 (socket says 172.21.15.92:52232) 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc2d9d640 1 -- 172.21.15.92:0/2218013620 learned_addr learned my addr 172.21.15.92:0/2218013620 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0edaf0 0x55a2ab1656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc359e640 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 0x55a2ab16c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc2d9d640 1 -- 172.21.15.92:0/2218013620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 msgr2=0x55a2ab16c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 0x55a2ab16c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc2d9d640 1 -- 172.21.15.92:0/2218013620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0edaf0 msgr2=0x55a2ab1656c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0edaf0 0x55a2ab1656c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc2d9d640 1 -- 172.21.15.92:0/2218013620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2ab16ccf0 con 0x55a2ab0ed100 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc259c640 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0edaf0 0x55a2ab1656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bc2d9d640 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0ed100 0x55a2ab165180 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009250 tx=0x7f4bac004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13db3867eb3bdc15 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:58:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4baa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2218013620 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac01d040 con 0x55a2ab0ed100 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2218013620 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f4bac009c10 con 0x55a2ab0ed100 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2218013620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb4059030 con 0x55a2ab0ed100 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2218013620 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bac004040 con 0x55a2ab0ed100 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 10 client.15804 fetch_fsmap learned FSMap version 29 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.616+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2218013620 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2ab16e000 con 0x55a2ab0ed100 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2218013620 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4059030 con 0x55a2ab0ed100 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2218013620 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4bac02c070 con 0x55a2ab0ed100 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4bcbc7e740 10 client.15804 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4bcbc7e740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4bcbc7e740 20 client.15804 populate_metadata read hostname 'smithi092' 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55a2ab102f20 0x55a2ab1708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4bcbc7e740 4 client.15804 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2218013620 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2ab059470 con 0x55a2ab102f20 2024-03-19T01:58:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4bc259c640 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55a2ab102f20 0x55a2ab1708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:58:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4bc259c640 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55a2ab102f20 0x55a2ab1708b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7bc8ad2178d66f4 server_cookie=4f1601089ee4fdad in_seq=0 out_seq=0 2024-03-19T01:58:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.617+0000 7f4baa7fc640 10 client.15804 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4baa7fc640 1 -- 172.21.15.92:0/2218013620 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2ab059470 con 0x55a2ab102f20 2024-03-19T01:58:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4baa7fc640 10 client.15804 handle_command_reply: tid=0 2024-03-19T01:58:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 client.15804 shutdown 2024-03-19T01:58:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2218013620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0ed100 msgr2=0x55a2ab165180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0ed100 0x55a2ab165180 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009250 tx=0x7f4bac004a70 comp rx=0 tx=0).stop 2024-03-19T01:58:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2218013620 shutdown_connections 2024-03-19T01:58:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55a2ab102f20 0x55a2ab1708b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2ab0f2c60 0x55a2ab16c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2ab0edaf0 0x55a2ab1656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 --2- 172.21.15.92:0/2218013620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2ab0ed100 0x55a2ab165180 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2218013620 >> 172.21.15.92:0/2218013620 conn(0x55a2ab0e8670 msgr2=0x55a2ab0ec160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4baa7fc640 0 client.15804 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2218013620 shutdown_connections 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 1 -- 172.21.15.92:0/2218013620 wait complete. 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.624+0000 7f4bcbc7e740 20 client.15804 trim_cache size 0 max 16384 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:58:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:58:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 -- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4bb00a28a0 msgr2=0x7f4bb00a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4bb00a28a0 0x7f4bb00a4d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4bb402c7b0 tx=0x7f4bb40095a0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 -- 172.21.15.92:0/1617813014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bc4103430 msgr2=0x7f4bc419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:58:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bc4103430 0x7f4bc419f070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009fd0 tx=0x7f4bbc00ef10 comp rx=0 tx=0).stop 2024-03-19T01:58:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 -- 172.21.15.92:0/1617813014 shutdown_connections 2024-03-19T01:58:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f4bb00a28a0 0x7f4bb00a4d60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bc4103430 0x7f4bc419f070 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bc4102a70 0x7f4bc4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 --2- 172.21.15.92:0/1617813014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bc4101880 0x7f4bc4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:58:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 -- 172.21.15.92:0/1617813014 >> 172.21.15.92:0/1617813014 conn(0x7f4bc40fcdf0 msgr2=0x7f4bc4105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:58:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 -- 172.21.15.92:0/1617813014 shutdown_connections 2024-03-19T01:58:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:58:49.625+0000 7f4bc908c640 1 -- 172.21.15.92:0/1617813014 wait complete. 2024-03-19T01:58:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:50 smithi159 ceph-mon[67110]: pgmap v532: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T01:58:51.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:50 smithi092 ceph-mon[62574]: pgmap v532: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T01:58:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:50 smithi144 ceph-mon[67199]: pgmap v532: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T01:58:52.924 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.924 INFO:tasks.workunit.client.0.smithi092.stdout:251/600: open fd = 3 2024-03-19T01:58:52.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.924 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.924 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.924 INFO:tasks.workunit.client.0.smithi092.stdout:252/600: open fd = 3 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:253/600: open fd = 3 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:254/600: open fd = 3 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:255/600: open fd = 3 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.925 INFO:tasks.workunit.client.0.smithi092.stdout:256/600: open fd = 3 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:257/600: open fd = 3 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:258/600: open fd = 3 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:259/600: open fd = 3 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.926 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:260/600: open fd = 3 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:261/600: open fd = 3 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:262/600: open fd = 3 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.927 INFO:tasks.workunit.client.0.smithi092.stdout:263/600: open fd = 3 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:264/600: open fd = 3 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:265/600: open fd = 3 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:266/600: open fd = 3 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.928 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:267/600: open fd = 3 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:268/600: open fd = 3 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:269/600: open fd = 3 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:270/600: open fd = 3 2024-03-19T01:58:52.929 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:271/600: open fd = 3 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:272/600: open fd = 3 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:273/600: open fd = 3 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.930 INFO:tasks.workunit.client.0.smithi092.stdout:274/600: open fd = 3 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:275/600: open fd = 3 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:276/600: open fd = 3 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:277/600: open fd = 3 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.931 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:278/600: open fd = 3 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:279/600: open fd = 3 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:280/600: open fd = 3 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:281/600: open fd = 3 2024-03-19T01:58:52.932 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:282/600: open fd = 3 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:283/600: open fd = 3 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:284/600: open fd = 3 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.933 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:285/600: open fd = 3 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:286/600: open fd = 3 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:287/600: open fd = 3 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:288/600: open fd = 3 2024-03-19T01:58:52.934 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:289/600: open fd = 3 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:290/600: open fd = 3 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:291/600: open fd = 3 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:292/600: open fd = 3 2024-03-19T01:58:52.935 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:293/600: open fd = 3 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:294/600: open fd = 3 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:295/600: open fd = 3 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.936 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:296/600: open fd = 3 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:297/600: open fd = 3 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:298/600: open fd = 3 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:299/600: open fd = 3 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.937 INFO:tasks.workunit.client.0.smithi092.stdout:300/600: open fd = 3 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:301/600: open fd = 3 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:302/600: open fd = 3 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:303/600: open fd = 3 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.938 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:304/600: open fd = 3 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:305/600: open fd = 3 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:306/600: open fd = 3 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:307/600: open fd = 3 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.939 INFO:tasks.workunit.client.0.smithi092.stdout:308/600: open fd = 3 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:309/600: open fd = 3 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:310/600: open fd = 3 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:311/600: open fd = 3 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:58:52.940 INFO:tasks.workunit.client.0.smithi092.stdout:312/600: open fd = 3 2024-03-19T01:58:52.941 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.941 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:58:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:52 smithi159 ceph-mon[67110]: pgmap v533: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:58:53.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:52 smithi092 ceph-mon[62574]: pgmap v533: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:58:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:52 smithi144 ceph-mon[67199]: pgmap v533: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:58:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:54 smithi159 ceph-mon[67110]: pgmap v534: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:55.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:54 smithi092 ceph-mon[62574]: pgmap v534: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:55.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:54 smithi144 ceph-mon[67199]: pgmap v534: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:58:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:56 smithi159 ceph-mon[67110]: pgmap v535: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:58:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:56 smithi092 ceph-mon[62574]: pgmap v535: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:58:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:56 smithi144 ceph-mon[67199]: pgmap v535: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:58:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:58:58 smithi159 ceph-mon[67110]: pgmap v536: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:58:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:58:58 smithi092 ceph-mon[62574]: pgmap v536: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:58:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:58:58 smithi144 ceph-mon[67199]: pgmap v536: 97 pgs: 97 active+clean; 2.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:59:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:00 smithi159 ceph-mon[67110]: pgmap v537: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:00 smithi092 ceph-mon[62574]: pgmap v537: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:00 smithi144 ceph-mon[67199]: pgmap v537: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 -- 172.21.15.92:0/1647834926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 msgr2=0x7f16e81046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 --2- 172.21.15.92:0/1647834926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 0x7f16e81046b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009900 tx=0x7f16e402bc30 comp rx=0 tx=0).stop 2024-03-19T01:59:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 -- 172.21.15.92:0/1647834926 shutdown_connections 2024-03-19T01:59:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 --2- 172.21.15.92:0/1647834926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16e8107520 0x7f16e8109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 --2- 172.21.15.92:0/1647834926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e8104bf0 0x7f16e8106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 --2- 172.21.15.92:0/1647834926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 0x7f16e81046b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 -- 172.21.15.92:0/1647834926 >> 172.21.15.92:0/1647834926 conn(0x7f16e80fbcb0 msgr2=0x7f16e80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 -- 172.21.15.92:0/1647834926 shutdown_connections 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 -- 172.21.15.92:0/1647834926 wait complete. 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 Processor -- start 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.716+0000 7f16efc85640 1 -- start start 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16efc85640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 0x7f16e8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16efc85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e8104bf0 0x7f16e8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16efc85640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16e8107520 0x7f16e819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16efc85640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16e810d4c0 con 0x7f16e81022c0 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16efc85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16e810d340 con 0x7f16e8104bf0 2024-03-19T01:59:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16efc85640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16e810d640 con 0x7f16e8107520 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16eda21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 0x7f16e8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16eda21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 0x7f16e8197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52912/0 (socket says 172.21.15.92:52912) 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16eda21640 1 -- 172.21.15.92:0/1970622846 learned_addr learned my addr 172.21.15.92:0/1970622846 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16ed220640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e8104bf0 0x7f16e8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16ee222640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16e8107520 0x7f16e819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16eda21640 1 -- 172.21.15.92:0/1970622846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16e8107520 msgr2=0x7f16e819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16eda21640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16e8107520 0x7f16e819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16eda21640 1 -- 172.21.15.92:0/1970622846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e8104bf0 msgr2=0x7f16e8198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16eda21640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e8104bf0 0x7f16e8198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16eda21640 1 -- 172.21.15.92:0/1970622846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e819f720 con 0x7f16e81022c0 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16eda21640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 0x7f16e8197b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009230 tx=0x7f16e402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16deffd640 1 -- 172.21.15.92:0/1970622846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4041070 con 0x7f16e81022c0 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16deffd640 1 -- 172.21.15.92:0/1970622846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e4004450 con 0x7f16e81022c0 2024-03-19T01:59:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16deffd640 1 -- 172.21.15.92:0/1970622846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4003db0 con 0x7f16e81022c0 2024-03-19T01:59:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16efc85640 1 -- 172.21.15.92:0/1970622846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e819f9b0 con 0x7f16e81022c0 2024-03-19T01:59:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.717+0000 7f16efc85640 1 -- 172.21.15.92:0/1970622846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e819fe50 con 0x7f16e81022c0 2024-03-19T01:59:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.719+0000 7f16deffd640 1 -- 172.21.15.92:0/1970622846 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f16e4044070 con 0x7f16e81022c0 2024-03-19T01:59:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16f287a740 1 Processor -- start 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16f287a740 1 -- start start 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16f287a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344ea9f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16f287a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9faf0 0x56344eaa45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16f287a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344eaa4c60 0x56344eaa70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16f287a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56344ea03be0 con 0x56344ea9f100 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16f287a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56344e9ac960 con 0x56344ea9faf0 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16f287a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56344e9ffc80 con 0x56344eaa4c60 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16ee222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344ea9f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16ee222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344ea9f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52914/0 (socket says 172.21.15.92:52914) 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16ee222640 1 -- 172.21.15.92:0/1929804996 learned_addr learned my addr 172.21.15.92:0/1929804996 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16ed220640 1 --2- 172.21.15.92:0/1929804996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344eaa4c60 0x56344eaa70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.720+0000 7f16eda21640 1 --2- 172.21.15.92:0/1929804996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9faf0 0x56344eaa45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16deffd640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16d40a28a0 0x7f16d40a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16ee222640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16d40a28a0 0x7f16d40a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16ed220640 1 -- 172.21.15.92:0/1929804996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9faf0 msgr2=0x56344eaa45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16ed220640 1 --2- 172.21.15.92:0/1929804996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9faf0 0x56344eaa45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16ed220640 1 -- 172.21.15.92:0/1929804996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 msgr2=0x56344ea9f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16ed220640 1 --2- 172.21.15.92:0/1929804996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344ea9f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16ed220640 1 -- 172.21.15.92:0/1929804996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344e9a3d20 con 0x56344eaa4c60 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16ed220640 1 --2- 172.21.15.92:0/1929804996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344eaa4c60 0x56344eaa70f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16e8069a00 tx=0x7f16d8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0cc9889cd629a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16d3fff640 1 -- 172.21.15.92:0/1929804996 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d801e070 con 0x56344eaa4c60 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16deffd640 1 -- 172.21.15.92:0/1970622846 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f16e40f0620 con 0x7f16e81022c0 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16ee222640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16d40a28a0 0x7f16d40a4d60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x56344eaa0350 tx=0x7f16e0004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16d3fff640 1 -- 172.21.15.92:0/1929804996 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8009d70 con 0x56344eaa4c60 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.721+0000 7f16d3fff640 1 -- 172.21.15.92:0/1929804996 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d80187b0 con 0x56344eaa4c60 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.722+0000 7f16f287a740 1 -- 172.21.15.92:0/1929804996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344eaa4c60 msgr2=0x56344eaa70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.722+0000 7f16f287a740 1 --2- 172.21.15.92:0/1929804996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344eaa4c60 0x56344eaa70f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f16e8069a00 tx=0x7f16d8011810 comp rx=0 tx=0).stop 2024-03-19T01:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.722+0000 7f16f287a740 1 -- 172.21.15.92:0/1929804996 shutdown_connections 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.722+0000 7f16f287a740 1 --2- 172.21.15.92:0/1929804996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344eaa4c60 0x56344eaa70f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.722+0000 7f16f287a740 1 --2- 172.21.15.92:0/1929804996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9faf0 0x56344eaa45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.722+0000 7f16f287a740 1 --2- 172.21.15.92:0/1929804996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344ea9f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.722+0000 7f16f287a740 1 -- 172.21.15.92:0/1929804996 >> 172.21.15.92:0/1929804996 conn(0x56344ea9a670 msgr2=0x56344ea9aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.722+0000 7f16f287a740 1 -- 172.21.15.92:0/1929804996 shutdown_connections 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.722+0000 7f16f287a740 1 -- 172.21.15.92:0/1929804996 wait complete. 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16f287a740 1 -- start start 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16f287a740 1 Processor -- start 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16f287a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 0x56344eaa9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16f287a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaa9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16f287a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eab0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16f287a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56344eaa7c10 con 0x56344eaa4c60 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16f287a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56344e9bfdb0 con 0x56344ea9f100 2024-03-19T01:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16f287a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56344eaa7d90 con 0x56344ea9faf0 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16f287a740 10 client.0 _ll_register_callbacks cb 0x56344e9c4e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16ee222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eab0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16ed220640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaa9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16ee222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eab0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52922/0 (socket says 172.21.15.92:52922) 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16ee222640 1 -- 172.21.15.92:0/2716468392 learned_addr learned my addr 172.21.15.92:0/2716468392 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.723+0000 7f16eda21640 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 0x56344eaa9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16ed220640 1 -- 172.21.15.92:0/2716468392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 msgr2=0x56344eaa9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16ed220640 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 0x56344eaa9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16ed220640 1 -- 172.21.15.92:0/2716468392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 msgr2=0x56344eab0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16ed220640 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eab0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16ed220640 1 -- 172.21.15.92:0/2716468392 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344e9bec60 con 0x56344ea9faf0 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16ee222640 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eab0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16ed220640 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaa9640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16d801c040 tx=0x7f16d80052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e2588e12be9918 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16d17fa640 1 -- 172.21.15.92:0/2716468392 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d801e040 con 0x56344ea9faf0 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16f287a740 1 -- 172.21.15.92:0/2716468392 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d8009d70 con 0x56344ea9faf0 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16d17fa640 1 -- 172.21.15.92:0/2716468392 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f16d8016040 con 0x56344ea9faf0 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16d17fa640 1 -- 172.21.15.92:0/2716468392 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16d8004490 con 0x56344ea9faf0 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16f287a740 10 client.25705 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16f287a740 1 -- 172.21.15.92:0/2716468392 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56344e718210 con 0x56344ea9faf0 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.724+0000 7f16d17fa640 1 -- 172.21.15.92:0/2716468392 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8009d70 con 0x56344ea9faf0 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16d17fa640 1 -- 172.21.15.92:0/2716468392 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f16d8019410 con 0x56344ea9faf0 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16f287a740 10 client.25705 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16f287a740 10 client.25705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16f287a740 20 client.25705 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16f287a740 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56344eab4ce0 0x56344eab7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16f287a740 4 client.25705 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16f287a740 1 -- 172.21.15.92:0/2716468392 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56344ea11120 con 0x56344eab4ce0 2024-03-19T01:59:02.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16eda21640 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56344eab4ce0 0x56344eab7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16d17fa640 10 client.25705 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.725+0000 7f16eda21640 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56344eab4ce0 0x56344eab7140 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef771709a93a4d03 server_cookie=3c9d6cee144049eb in_seq=0 out_seq=0 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16d17fa640 1 -- 172.21.15.92:0/2716468392 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56344ea11120 con 0x56344eab4ce0 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16d17fa640 10 client.25705 handle_command_reply: tid=0 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 client.25705 shutdown 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 -- 172.21.15.92:0/2716468392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 msgr2=0x56344eaa9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaa9640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16d801c040 tx=0x7f16d80052f0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 -- 172.21.15.92:0/2716468392 shutdown_connections 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56344eab4ce0 0x56344eab7140 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eab0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaa9640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 --2- 172.21.15.92:0/2716468392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 0x56344eaa9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 -- 172.21.15.92:0/2716468392 >> 172.21.15.92:0/2716468392 conn(0x56344ea9a670 msgr2=0x56344ea9b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:02.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16d17fa640 0 client.25705 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 -- 172.21.15.92:0/2716468392 shutdown_connections 2024-03-19T01:59:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 1 -- 172.21.15.92:0/2716468392 wait complete. 2024-03-19T01:59:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.726+0000 7f16f287a740 20 client.25705 trim_cache size 0 max 16384 2024-03-19T01:59:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16f287a740 1 Processor -- start 2024-03-19T01:59:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16f287a740 1 -- start start 2024-03-19T01:59:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16f287a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 0x56344eaa0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16f287a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eab0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16f287a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eaa08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16f287a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a8c90 con 0x56344eaa4c60 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16f287a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81aa7d0 con 0x56344ea9f100 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16f287a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a9360 con 0x56344ea9faf0 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16ee222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eaa08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16ee222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eaa08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52934/0 (socket says 172.21.15.92:52934) 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16ed220640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eab0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16ee222640 1 -- 172.21.15.92:0/2701803465 learned_addr learned my addr 172.21.15.92:0/2701803465 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.741+0000 7f16eda21640 1 --2- 172.21.15.92:0/2701803465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 0x56344eaa0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16ed220640 1 -- 172.21.15.92:0/2701803465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 msgr2=0x56344eaa0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:02.750 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16ed220640 1 --2- 172.21.15.92:0/2701803465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 0x56344eaa0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16ed220640 1 -- 172.21.15.92:0/2701803465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 msgr2=0x56344eaa08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16ed220640 1 --2- 172.21.15.92:0/2701803465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eaa08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16ed220640 1 -- 172.21.15.92:0/2701803465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344e7e4620 con 0x56344ea9faf0 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16ee222640 1 --2- 172.21.15.92:0/2701803465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eaa08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16ed220640 1 --2- 172.21.15.92:0/2701803465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eab0640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f16d8005e00 tx=0x7f16d8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3a907dc2de54331 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16d37fe640 1 -- 172.21.15.92:0/2701803465 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d801e040 con 0x56344ea9faf0 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16d37fe640 1 -- 172.21.15.92:0/2701803465 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8009cb0 con 0x56344ea9faf0 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16d37fe640 1 -- 172.21.15.92:0/2701803465 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d8004630 con 0x56344ea9faf0 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16f287a740 1 -- 172.21.15.92:0/2701803465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 msgr2=0x56344eab0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16f287a740 1 --2- 172.21.15.92:0/2701803465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eab0640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f16d8005e00 tx=0x7f16d8011810 comp rx=0 tx=0).stop 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16f287a740 1 -- 172.21.15.92:0/2701803465 shutdown_connections 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16f287a740 1 --2- 172.21.15.92:0/2701803465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344eaa4c60 0x56344eaa08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16f287a740 1 --2- 172.21.15.92:0/2701803465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eab0640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16f287a740 1 --2- 172.21.15.92:0/2701803465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344ea9f100 0x56344eaa0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.751 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.742+0000 7f16f287a740 1 -- 172.21.15.92:0/2701803465 >> 172.21.15.92:0/2701803465 conn(0x56344ea9a670 msgr2=0x56344ea99220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 -- 172.21.15.92:0/2701803465 shutdown_connections 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 -- 172.21.15.92:0/2701803465 wait complete. 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 -- start start 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 Processor -- start 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344eaaddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344eaa4c60 0x56344eaaf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81aa100 con 0x56344ea9f100 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a9a30 con 0x56344eaa4c60 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81a6c70 con 0x56344ea9faf0 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16f287a740 10 client.0 _ll_register_callbacks cb 0x7f16d801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16eda21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344eaaddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16eda21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344eaaddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52944/0 (socket says 172.21.15.92:52944) 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16eda21640 1 -- 172.21.15.92:0/2984454501 learned_addr learned my addr 172.21.15.92:0/2984454501 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16ed220640 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.743+0000 7f16ee222640 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344eaa4c60 0x56344eaaf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16eda21640 1 -- 172.21.15.92:0/2984454501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 msgr2=0x56344eaae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16eda21640 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16eda21640 1 -- 172.21.15.92:0/2984454501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344eaa4c60 msgr2=0x56344eaaf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16eda21640 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344eaa4c60 0x56344eaaf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16eda21640 1 -- 172.21.15.92:0/2984454501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344eaaf930 con 0x56344ea9f100 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16ed220640 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16eda21640 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344eaaddf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f16e40ee130 tx=0x7f16e40dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594ec1ea1e006f7e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16d17fa640 1 -- 172.21.15.92:0/2984454501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e404b040 con 0x56344ea9f100 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16d17fa640 1 -- 172.21.15.92:0/2984454501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f16e4004c60 con 0x56344ea9f100 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16f287a740 1 -- 172.21.15.92:0/2984454501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d8009cb0 con 0x56344ea9f100 2024-03-19T01:59:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16d17fa640 1 -- 172.21.15.92:0/2984454501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16e40bc930 con 0x56344ea9f100 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16f287a740 10 client.15822 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16f287a740 1 -- 172.21.15.92:0/2984454501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56344eb1b1d0 con 0x56344ea9f100 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16d17fa640 1 -- 172.21.15.92:0/2984454501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d8009cb0 con 0x56344ea9f100 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16d17fa640 1 -- 172.21.15.92:0/2984454501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f16e40b9640 con 0x56344ea9f100 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16f287a740 10 client.15822 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16f287a740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16f287a740 20 client.15822 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16f287a740 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56344eb1bb70 0x56344eb1df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16f287a740 4 client.15822 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:59:02.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.744+0000 7f16f287a740 1 -- 172.21.15.92:0/2984454501 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56344ea0b470 con 0x56344eb1bb70 2024-03-19T01:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.745+0000 7f16ed220640 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56344eb1bb70 0x56344eb1df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.745+0000 7f16d17fa640 10 client.15822 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.745+0000 7f16ed220640 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56344eb1bb70 0x56344eb1df90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66211340726dd3b server_cookie=3a3cda35f941f80 in_seq=0 out_seq=0 2024-03-19T01:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16d17fa640 1 -- 172.21.15.92:0/2984454501 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56344ea0b470 con 0x56344eb1bb70 2024-03-19T01:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16d17fa640 10 client.15822 handle_command_reply: tid=0 2024-03-19T01:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 client.15822 shutdown 2024-03-19T01:59:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 -- 172.21.15.92:0/2984454501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 msgr2=0x56344eaaddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344eaaddf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f16e40ee130 tx=0x7f16e40dd040 comp rx=0 tx=0).stop 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 -- 172.21.15.92:0/2984454501 shutdown_connections 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56344eb1bb70 0x56344eb1df90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56344eaa4c60 0x56344eaaf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344ea9faf0 0x56344eaae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 --2- 172.21.15.92:0/2984454501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56344ea9f100 0x56344eaaddf0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16d17fa640 0 client.15822 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 -- 172.21.15.92:0/2984454501 >> 172.21.15.92:0/2984454501 conn(0x56344ea9a670 msgr2=0x56344ea9d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 -- 172.21.15.92:0/2984454501 shutdown_connections 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 1 -- 172.21.15.92:0/2984454501 wait complete. 2024-03-19T01:59:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16f287a740 20 client.15822 trim_cache size 0 max 16384 2024-03-19T01:59:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:59:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:59:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:59:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16efc85640 1 -- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16d40a28a0 msgr2=0x7f16d40a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.752+0000 7f16efc85640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16d40a28a0 0x7f16d40a4d60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x56344eaa0350 tx=0x7f16e0004ab0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 -- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 msgr2=0x7f16e8197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 0x7f16e8197b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009230 tx=0x7f16e402c650 comp rx=0 tx=0).stop 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 -- 172.21.15.92:0/1970622846 shutdown_connections 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16d40a28a0 0x7f16d40a4d60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16e8107520 0x7f16e819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16e8104bf0 0x7f16e8198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 --2- 172.21.15.92:0/1970622846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e81022c0 0x7f16e8197b00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 -- 172.21.15.92:0/1970622846 >> 172.21.15.92:0/1970622846 conn(0x7f16e80fbcb0 msgr2=0x7f16e8102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 -- 172.21.15.92:0/1970622846 shutdown_connections 2024-03-19T01:59:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:02.753+0000 7f16efc85640 1 -- 172.21.15.92:0/1970622846 wait complete. 2024-03-19T01:59:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:02 smithi159 ceph-mon[67110]: pgmap v538: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:02 smithi092 ceph-mon[62574]: pgmap v538: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:02 smithi144 ceph-mon[67199]: pgmap v538: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:04 smithi159 ceph-mon[67110]: pgmap v539: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:04 smithi092 ceph-mon[62574]: pgmap v539: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:04 smithi144 ceph-mon[67199]: pgmap v539: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:06 smithi159 ceph-mon[67110]: pgmap v540: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T01:59:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:06 smithi092 ceph-mon[62574]: pgmap v540: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T01:59:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:06 smithi144 ceph-mon[67199]: pgmap v540: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T01:59:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:08 smithi159 ceph-mon[67110]: pgmap v541: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:59:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:08 smithi092 ceph-mon[62574]: pgmap v541: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:59:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:08 smithi144 ceph-mon[67199]: pgmap v541: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T01:59:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:10 smithi159 ceph-mon[67110]: pgmap v542: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:10 smithi092 ceph-mon[62574]: pgmap v542: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:10 smithi144 ceph-mon[67199]: pgmap v542: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:12 smithi159 ceph-mon[67110]: pgmap v543: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:59:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:12 smithi092 ceph-mon[62574]: pgmap v543: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:59:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:12 smithi144 ceph-mon[67199]: pgmap v543: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:59:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:14 smithi159 ceph-mon[67110]: pgmap v544: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:14 smithi092 ceph-mon[62574]: pgmap v544: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:14 smithi144 ceph-mon[67199]: pgmap v544: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.839+0000 7f97fc797640 1 -- 172.21.15.92:0/3207555907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f41022f0 msgr2=0x7f97f4108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.839+0000 7f97fc797640 1 --2- 172.21.15.92:0/3207555907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f41022f0 0x7f97f4108b80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009ca0 tx=0x7f97ec02c0f0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.839+0000 7f97fc797640 1 -- 172.21.15.92:0/3207555907 shutdown_connections 2024-03-19T01:59:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.839+0000 7f97fc797640 1 --2- 172.21.15.92:0/3207555907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f41022f0 0x7f97f4108b80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.839+0000 7f97fc797640 1 --2- 172.21.15.92:0/3207555907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97f4101930 0x7f97f4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.839+0000 7f97fc797640 1 --2- 172.21.15.92:0/3207555907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97f4100740 0x7f97f4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.839+0000 7f97fc797640 1 -- 172.21.15.92:0/3207555907 >> 172.21.15.92:0/3207555907 conn(0x7f97f40fbcb0 msgr2=0x7f97f40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.839+0000 7f97fc797640 1 -- 172.21.15.92:0/3207555907 shutdown_connections 2024-03-19T01:59:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.839+0000 7f97fc797640 1 -- 172.21.15.92:0/3207555907 wait complete. 2024-03-19T01:59:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fc797640 1 Processor -- start 2024-03-19T01:59:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fc797640 1 -- start start 2024-03-19T01:59:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fc797640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f4100740 0x7f97f4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fc797640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97f4101930 0x7f97f4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fc797640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97f41022f0 0x7f97f419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fc797640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97f410d4e0 con 0x7f97f4100740 2024-03-19T01:59:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fc797640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97f410d360 con 0x7f97f41022f0 2024-03-19T01:59:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fc797640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97f410d660 con 0x7f97f4101930 2024-03-19T01:59:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fa533640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f4100740 0x7f97f4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fa533640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f4100740 0x7f97f4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48762/0 (socket says 172.21.15.92:48762) 2024-03-19T01:59:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fa533640 1 -- 172.21.15.92:0/4271006799 learned_addr learned my addr 172.21.15.92:0/4271006799 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97f9d32640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97f4101930 0x7f97f4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fad34640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97f41022f0 0x7f97f419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fa533640 1 -- 172.21.15.92:0/4271006799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97f4101930 msgr2=0x7f97f4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fa533640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97f4101930 0x7f97f4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fa533640 1 -- 172.21.15.92:0/4271006799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97f41022f0 msgr2=0x7f97f419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fa533640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97f41022f0 0x7f97f419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fa533640 1 -- 172.21.15.92:0/4271006799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f419f6b0 con 0x7f97f4100740 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fad34640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97f41022f0 0x7f97f419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97fa533640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f4100740 0x7f97f4197ae0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f97e800e9c0 tx=0x7f97e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97e77fe640 1 -- 172.21.15.92:0/4271006799 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e800cd80 con 0x7f97f4100740 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97e77fe640 1 -- 172.21.15.92:0/4271006799 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e80035f0 con 0x7f97f4100740 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.840+0000 7f97e77fe640 1 -- 172.21.15.92:0/4271006799 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8010690 con 0x7f97f4100740 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.841+0000 7f97fc797640 1 -- 172.21.15.92:0/4271006799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f419f9a0 con 0x7f97f4100740 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.841+0000 7f97fc797640 1 -- 172.21.15.92:0/4271006799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f419fea0 con 0x7f97f4100740 2024-03-19T01:59:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.842+0000 7f97e77fe640 1 -- 172.21.15.92:0/4271006799 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f97e80037a0 con 0x7f97f4100740 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.843+0000 7f97e77fe640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f97e00a28a0 0x7f97e00a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.843+0000 7f97ff385740 1 Processor -- start 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.843+0000 7f97f9d32640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f97e00a28a0 0x7f97e00a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.843+0000 7f97e77fe640 1 -- 172.21.15.92:0/4271006799 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f97e80ceeb0 con 0x7f97f4100740 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.843+0000 7f97ff385740 1 -- start start 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97ff385740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97ff385740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 0x55ffa05f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97ff385740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 0x55ffa05f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97ff385740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ffa0555be0 con 0x55ffa05f1100 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97ff385740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ffa04fe960 con 0x55ffa05f6c60 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97ff385740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ffa0551c80 con 0x55ffa05f1af0 2024-03-19T01:59:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97f9d32640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f97e00a28a0 0x7f97e00a4d60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f97f4198fa0 tx=0x7f97f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fa533640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fa533640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05f1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48774/0 (socket says 172.21.15.92:48774) 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fa533640 1 -- 172.21.15.92:0/294764870 learned_addr learned my addr 172.21.15.92:0/294764870 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fad34640 1 --2- 172.21.15.92:0/294764870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 0x55ffa05f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97f9d32640 1 --2- 172.21.15.92:0/294764870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 0x55ffa05f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fa533640 1 -- 172.21.15.92:0/294764870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 msgr2=0x55ffa05f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fa533640 1 --2- 172.21.15.92:0/294764870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 0x55ffa05f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fa533640 1 -- 172.21.15.92:0/294764870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 msgr2=0x55ffa05f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fa533640 1 --2- 172.21.15.92:0/294764870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 0x55ffa05f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fa533640 1 -- 172.21.15.92:0/294764870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa04f5d20 con 0x55ffa05f1100 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97fa533640 1 --2- 172.21.15.92:0/294764870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05f1520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f97e809b7f0 tx=0x7f97e809b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4887620cbe500e1b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97e4ff9640 1 -- 172.21.15.92:0/294764870 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80b3020 con 0x55ffa05f1100 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97e4ff9640 1 -- 172.21.15.92:0/294764870 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e809f9b0 con 0x55ffa05f1100 2024-03-19T01:59:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.844+0000 7f97e4ff9640 1 -- 172.21.15.92:0/294764870 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80a3b70 con 0x55ffa05f1100 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.845+0000 7f97ff385740 1 -- 172.21.15.92:0/294764870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 msgr2=0x55ffa05f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.845+0000 7f97ff385740 1 --2- 172.21.15.92:0/294764870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05f1520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f97e809b7f0 tx=0x7f97e809b760 comp rx=0 tx=0).stop 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.845+0000 7f97ff385740 1 -- 172.21.15.92:0/294764870 shutdown_connections 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.845+0000 7f97ff385740 1 --2- 172.21.15.92:0/294764870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 0x55ffa05f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.845+0000 7f97ff385740 1 --2- 172.21.15.92:0/294764870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 0x55ffa05f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.845+0000 7f97ff385740 1 --2- 172.21.15.92:0/294764870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05f1520 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.845+0000 7f97ff385740 1 -- 172.21.15.92:0/294764870 >> 172.21.15.92:0/294764870 conn(0x55ffa05ec670 msgr2=0x55ffa05eca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.845+0000 7f97ff385740 1 -- 172.21.15.92:0/294764870 shutdown_connections 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.845+0000 7f97ff385740 1 -- 172.21.15.92:0/294764870 wait complete. 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97ff385740 1 -- start start 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97ff385740 1 Processor -- start 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97ff385740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1100 0x55ffa05fb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97ff385740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1af0 0x55ffa05fb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97ff385740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 0x55ffa06025a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97ff385740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ffa05f9bb0 con 0x55ffa05f1af0 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97ff385740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ffa0511db0 con 0x55ffa05f1100 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97ff385740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ffa05f9d30 con 0x55ffa05f6c60 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97ff385740 10 client.0 _ll_register_callbacks cb 0x55ffa0516e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97f9d32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1af0 0x55ffa05fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97f9d32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1af0 0x55ffa05fb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48788/0 (socket says 172.21.15.92:48788) 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97f9d32640 1 -- 172.21.15.92:0/74077781 learned_addr learned my addr 172.21.15.92:0/74077781 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97fa533640 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1100 0x55ffa05fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97fad34640 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 0x55ffa06025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97f9d32640 1 -- 172.21.15.92:0/74077781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 msgr2=0x55ffa06025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97f9d32640 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 0x55ffa06025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97f9d32640 1 -- 172.21.15.92:0/74077781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1100 msgr2=0x55ffa05fb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97f9d32640 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1100 0x55ffa05fb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97f9d32640 1 -- 172.21.15.92:0/74077781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa0510c60 con 0x55ffa05f1af0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.846+0000 7f97fa533640 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1100 0x55ffa05fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97f9d32640 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1af0 0x55ffa05fb560 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f97f0003a40 tx=0x7f97f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f8f11973582239 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ddffb640 1 -- 172.21.15.92:0/74077781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0005020 con 0x55ffa05f1af0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ddffb640 1 -- 172.21.15.92:0/74077781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f97f00051c0 con 0x55ffa05f1af0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ff385740 1 -- 172.21.15.92:0/74077781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e809f9b0 con 0x55ffa05f1af0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ddffb640 1 -- 172.21.15.92:0/74077781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97f00053c0 con 0x55ffa05f1af0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ff385740 10 client.15834 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ff385740 1 -- 172.21.15.92:0/74077781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffa026a210 con 0x55ffa05f1af0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ddffb640 1 -- 172.21.15.92:0/74077781 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e809f9b0 con 0x55ffa05f1af0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ddffb640 1 -- 172.21.15.92:0/74077781 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f97f0026070 con 0x55ffa05f1af0 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ff385740 10 client.15834 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ff385740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ff385740 20 client.15834 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ff385740 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ffa0606eb0 0x55ffa0609350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ff385740 4 client.15834 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.847+0000 7f97ff385740 1 -- 172.21.15.92:0/74077781 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa0563120 con 0x55ffa0606eb0 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.848+0000 7f97fa533640 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ffa0606eb0 0x55ffa0609350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.848+0000 7f97ddffb640 10 client.15834 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.848+0000 7f97fa533640 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ffa0606eb0 0x55ffa0609350 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a57d741288abbbca server_cookie=f6a3bce515019852 in_seq=0 out_seq=0 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ddffb640 1 -- 172.21.15.92:0/74077781 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ffa0563120 con 0x55ffa0606eb0 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ddffb640 10 client.15834 handle_command_reply: tid=0 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 client.15834 shutdown 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 -- 172.21.15.92:0/74077781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1af0 msgr2=0x55ffa05fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1af0 0x55ffa05fb560 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f97f0003a40 tx=0x7f97f0004730 comp rx=0 tx=0).stop 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 -- 172.21.15.92:0/74077781 shutdown_connections 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ffa0606eb0 0x55ffa0609350 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 0x55ffa06025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ddffb640 0 client.15834 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1af0 0x55ffa05fb560 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 --2- 172.21.15.92:0/74077781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1100 0x55ffa05fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 -- 172.21.15.92:0/74077781 >> 172.21.15.92:0/74077781 conn(0x55ffa05ec670 msgr2=0x55ffa05ed8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 -- 172.21.15.92:0/74077781 shutdown_connections 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 1 -- 172.21.15.92:0/74077781 wait complete. 2024-03-19T01:59:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.849+0000 7f97ff385740 20 client.15834 trim_cache size 0 max 16384 2024-03-19T01:59:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.863+0000 7f97ff385740 1 Processor -- start 2024-03-19T01:59:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.863+0000 7f97ff385740 1 -- start start 2024-03-19T01:59:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.863+0000 7f97ff385740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05effe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.863+0000 7f97ff385740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 0x55ffa060dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.863+0000 7f97ff385740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 0x55ffa060e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.863+0000 7f97ff385740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41a8d30 con 0x55ffa05f1100 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.863+0000 7f97ff385740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41aa870 con 0x55ffa05f6c60 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.863+0000 7f97ff385740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41a9400 con 0x55ffa05f1af0 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fa533640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fa533640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05effe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48804/0 (socket says 172.21.15.92:48804) 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fa533640 1 -- 172.21.15.92:0/3821554992 learned_addr learned my addr 172.21.15.92:0/3821554992 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97f9d32640 1 --2- 172.21.15.92:0/3821554992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 0x55ffa060dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fad34640 1 --2- 172.21.15.92:0/3821554992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 0x55ffa060e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fa533640 1 -- 172.21.15.92:0/3821554992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 msgr2=0x55ffa060dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fa533640 1 --2- 172.21.15.92:0/3821554992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 0x55ffa060dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fa533640 1 -- 172.21.15.92:0/3821554992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 msgr2=0x55ffa060e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fa533640 1 --2- 172.21.15.92:0/3821554992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 0x55ffa060e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fa533640 1 -- 172.21.15.92:0/3821554992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa0336620 con 0x55ffa05f1100 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97fa533640 1 --2- 172.21.15.92:0/3821554992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05effe0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f97e800c6a0 tx=0x7f97e809c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3306e22e45bbe495 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97dffff640 1 -- 172.21.15.92:0/3821554992 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8064020 con 0x55ffa05f1100 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97dffff640 1 -- 172.21.15.92:0/3821554992 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e809f9b0 con 0x55ffa05f1100 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.864+0000 7f97dffff640 1 -- 172.21.15.92:0/3821554992 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80b3040 con 0x55ffa05f1100 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 -- 172.21.15.92:0/3821554992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 msgr2=0x55ffa05effe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 --2- 172.21.15.92:0/3821554992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05effe0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f97e800c6a0 tx=0x7f97e809c0a0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 -- 172.21.15.92:0/3821554992 shutdown_connections 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 --2- 172.21.15.92:0/3821554992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f6c60 0x55ffa060e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 --2- 172.21.15.92:0/3821554992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f1af0 0x55ffa060dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 --2- 172.21.15.92:0/3821554992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa05effe0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 -- 172.21.15.92:0/3821554992 >> 172.21.15.92:0/3821554992 conn(0x55ffa05ec670 msgr2=0x55ffa05ef8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 -- 172.21.15.92:0/3821554992 shutdown_connections 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 -- 172.21.15.92:0/3821554992 wait complete. 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 -- start start 2024-03-19T01:59:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 Processor -- start 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa0668af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1af0 0x55ffa0669030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 0x55ffa0670070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41aa1a0 con 0x55ffa05f1100 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41a9ad0 con 0x55ffa05f1af0 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41a6d10 con 0x55ffa05f6c60 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97ff385740 10 client.0 _ll_register_callbacks cb 0x7f97f0004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97fa533640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa0668af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97fa533640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa0668af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48814/0 (socket says 172.21.15.92:48814) 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.865+0000 7f97fa533640 1 -- 172.21.15.92:0/3612320470 learned_addr learned my addr 172.21.15.92:0/3612320470 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97fad34640 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 0x55ffa0670070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97f9d32640 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1af0 0x55ffa0669030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97fa533640 1 -- 172.21.15.92:0/3612320470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 msgr2=0x55ffa0670070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97fa533640 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 0x55ffa0670070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97fa533640 1 -- 172.21.15.92:0/3612320470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1af0 msgr2=0x55ffa0669030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97fa533640 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1af0 0x55ffa0669030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97fa533640 1 -- 172.21.15.92:0/3612320470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa0670660 con 0x55ffa05f1100 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97fa533640 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa0668af0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f97e80058c0 tx=0x7f97e80097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4446651e4218e14d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97ddffb640 1 -- 172.21.15.92:0/3612320470 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80b3030 con 0x55ffa05f1100 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97ddffb640 1 -- 172.21.15.92:0/3612320470 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f97e80a3d70 con 0x55ffa05f1100 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97ff385740 1 -- 172.21.15.92:0/3612320470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e809f9b0 con 0x55ffa05f1100 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97ddffb640 1 -- 172.21.15.92:0/3612320470 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97e8064070 con 0x55ffa05f1100 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97ff385740 10 client.15840 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97ff385740 1 -- 172.21.15.92:0/3612320470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffa0671970 con 0x55ffa05f1100 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97ddffb640 1 -- 172.21.15.92:0/3612320470 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e809f9b0 con 0x55ffa05f1100 2024-03-19T01:59:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.866+0000 7f97ddffb640 1 -- 172.21.15.92:0/3612320470 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f97e809a720 con 0x55ffa05f1100 2024-03-19T01:59:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.867+0000 7f97ff385740 10 client.15840 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.867+0000 7f97ff385740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.867+0000 7f97ff385740 20 client.15840 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.867+0000 7f97ff385740 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ffa0671f90 0x55ffa0674380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.867+0000 7f97ff385740 4 client.15840 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:59:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.867+0000 7f97ff385740 1 -- 172.21.15.92:0/3612320470 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffa055d470 con 0x55ffa0671f90 2024-03-19T01:59:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.867+0000 7f97f9d32640 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ffa0671f90 0x55ffa0674380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.867+0000 7f97ddffb640 10 client.15840 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.867+0000 7f97f9d32640 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ffa0671f90 0x55ffa0674380 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b324b72d125bbef4 server_cookie=17ad9394aab38a5 in_seq=0 out_seq=0 2024-03-19T01:59:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.873+0000 7f97ddffb640 1 -- 172.21.15.92:0/3612320470 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ffa055d470 con 0x55ffa0671f90 2024-03-19T01:59:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.873+0000 7f97ddffb640 10 client.15840 handle_command_reply: tid=0 2024-03-19T01:59:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.873+0000 7f97ff385740 1 client.15840 shutdown 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 -- 172.21.15.92:0/3612320470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 msgr2=0x55ffa0668af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa0668af0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f97e80058c0 tx=0x7f97e80097b0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 -- 172.21.15.92:0/3612320470 shutdown_connections 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ffa0671f90 0x55ffa0674380 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa05f6c60 0x55ffa0670070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffa05f1af0 0x55ffa0669030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 --2- 172.21.15.92:0/3612320470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ffa05f1100 0x55ffa0668af0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ddffb640 0 client.15840 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 -- 172.21.15.92:0/3612320470 >> 172.21.15.92:0/3612320470 conn(0x55ffa05ec670 msgr2=0x55ffa05f0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 -- 172.21.15.92:0/3612320470 shutdown_connections 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 1 -- 172.21.15.92:0/3612320470 wait complete. 2024-03-19T01:59:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97ff385740 20 client.15840 trim_cache size 0 max 16384 2024-03-19T01:59:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:59:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:59:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:59:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97fc797640 1 -- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f97e00a28a0 msgr2=0x7f97e00a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97fc797640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f97e00a28a0 0x7f97e00a4d60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f97f4198fa0 tx=0x7f97f0005f90 comp rx=0 tx=0).stop 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97fc797640 1 -- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f4100740 msgr2=0x7f97f4197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97fc797640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f4100740 0x7f97f4197ae0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f97e800e9c0 tx=0x7f97e800ee80 comp rx=0 tx=0).stop 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97fc797640 1 -- 172.21.15.92:0/4271006799 shutdown_connections 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97fc797640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97f41022f0 0x7f97f419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97fc797640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f97e00a28a0 0x7f97e00a4d60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97fc797640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97f4101930 0x7f97f4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.874+0000 7f97fc797640 1 --2- 172.21.15.92:0/4271006799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97f4100740 0x7f97f4197ae0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.875+0000 7f97fc797640 1 -- 172.21.15.92:0/4271006799 >> 172.21.15.92:0/4271006799 conn(0x7f97f40fbcb0 msgr2=0x7f97f4104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.875+0000 7f97fc797640 1 -- 172.21.15.92:0/4271006799 shutdown_connections 2024-03-19T01:59:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:15.875+0000 7f97fc797640 1 -- 172.21.15.92:0/4271006799 wait complete. 2024-03-19T01:59:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:16 smithi159 ceph-mon[67110]: pgmap v545: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:59:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:16 smithi092 ceph-mon[62574]: pgmap v545: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:59:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:16 smithi144 ceph-mon[67199]: pgmap v545: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-03-19T01:59:17.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:17 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:59:18.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:17 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:59:18.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:17 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T01:59:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: pgmap v546: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2024-03-19T01:59:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-03-19T01:59:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: Unable to set osd_memory_target on smithi092 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-03-19T01:59:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:59:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:59:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:59:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: pgmap v546: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2024-03-19T01:59:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-03-19T01:59:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: Unable to set osd_memory_target on smithi092 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-03-19T01:59:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:59:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:59:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:59:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: pgmap v546: 97 pgs: 97 active+clean; 2.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2024-03-19T01:59:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-19T01:59:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-03-19T01:59:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: Unable to set osd_memory_target on smithi092 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-03-19T01:59:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T01:59:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T01:59:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T01:59:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:20 smithi159 ceph-mon[67110]: pgmap v547: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:59:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:20 smithi092 ceph-mon[62574]: pgmap v547: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:59:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:20 smithi144 ceph-mon[67199]: pgmap v547: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:59:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:22 smithi159 ceph-mon[67110]: pgmap v548: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:59:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:22 smithi092 ceph-mon[62574]: pgmap v548: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:59:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:22 smithi144 ceph-mon[67199]: pgmap v548: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T01:59:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:24 smithi159 ceph-mon[67110]: pgmap v549: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:25.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:24 smithi092 ceph-mon[62574]: pgmap v549: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:25.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:24 smithi144 ceph-mon[67199]: pgmap v549: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T01:59:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:26 smithi159 ceph-mon[67110]: pgmap v550: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:26 smithi092 ceph-mon[62574]: pgmap v550: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:26 smithi144 ceph-mon[67199]: pgmap v550: 97 pgs: 97 active+clean; 2.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:28 smithi159 ceph-mon[67110]: pgmap v551: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.961+0000 7fd88ab90640 1 -- 172.21.15.92:0/4062428912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 msgr2=0x7fd884101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.961+0000 7fd88ab90640 1 --2- 172.21.15.92:0/4062428912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 0x7fd884101db0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd874009900 tx=0x7fd87402bc30 comp rx=0 tx=0).stop 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.961+0000 7fd88ab90640 1 -- 172.21.15.92:0/4062428912 shutdown_connections 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.961+0000 7fd88ab90640 1 --2- 172.21.15.92:0/4062428912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8841022f0 0x7fd884108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.961+0000 7fd88ab90640 1 --2- 172.21.15.92:0/4062428912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 0x7fd884101db0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.961+0000 7fd88ab90640 1 --2- 172.21.15.92:0/4062428912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884100740 0x7fd884100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.961+0000 7fd88ab90640 1 -- 172.21.15.92:0/4062428912 >> 172.21.15.92:0/4062428912 conn(0x7fd8840fbcb0 msgr2=0x7fd8840fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.961+0000 7fd88ab90640 1 -- 172.21.15.92:0/4062428912 shutdown_connections 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.961+0000 7fd88ab90640 1 -- 172.21.15.92:0/4062428912 wait complete. 2024-03-19T01:59:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88ab90640 1 Processor -- start 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88ab90640 1 -- start start 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88ab90640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884100740 0x7fd884197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88ab90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 0x7fd884197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88ab90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8841022f0 0x7fd88419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88ab90640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd88410c3a0 con 0x7fd884101930 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88ab90640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd88410c220 con 0x7fd8841022f0 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88ab90640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd88410c520 con 0x7fd884100740 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 0x7fd884197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88892c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884100740 0x7fd884197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 0x7fd884197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50586/0 (socket says 172.21.15.92:50586) 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88892c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884100740 0x7fd884197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:36576/0 (socket says 172.21.15.92:36576) 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd87bfff640 1 -- 172.21.15.92:0/3558253777 learned_addr learned my addr 172.21.15.92:0/3558253777 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd88912d640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8841022f0 0x7fd88419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd87bfff640 1 -- 172.21.15.92:0/3558253777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884100740 msgr2=0x7fd884197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd87bfff640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884100740 0x7fd884197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd87bfff640 1 -- 172.21.15.92:0/3558253777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8841022f0 msgr2=0x7fd88419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd87bfff640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8841022f0 0x7fd88419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.962+0000 7fd87bfff640 1 -- 172.21.15.92:0/3558253777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88419f6d0 con 0x7fd884101930 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.963+0000 7fd87bfff640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 0x7fd884197ff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd874009230 tx=0x7fd87402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.963+0000 7fd879ffb640 1 -- 172.21.15.92:0/3558253777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874041070 con 0x7fd884101930 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.963+0000 7fd879ffb640 1 -- 172.21.15.92:0/3558253777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd874004450 con 0x7fd884101930 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.963+0000 7fd879ffb640 1 -- 172.21.15.92:0/3558253777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874003e20 con 0x7fd884101930 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.963+0000 7fd88ab90640 1 -- 172.21.15.92:0/3558253777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88419f960 con 0x7fd884101930 2024-03-19T01:59:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.963+0000 7fd88ab90640 1 -- 172.21.15.92:0/3558253777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88419fe00 con 0x7fd884101930 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd879ffb640 1 -- 172.21.15.92:0/3558253777 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fd874044070 con 0x7fd884101930 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88d780740 1 Processor -- start 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88d780740 1 -- start start 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88d780740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 0x55ebbfd2c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88d780740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd31a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88d780740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd32070 0x55ebbfd34500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88d780740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ebbfc8cbe0 con 0x55ebbfd2cf00 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88d780740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ebbfc35960 con 0x55ebbfd32070 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88d780740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ebbfc88c80 con 0x55ebbfd2c510 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88892c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd31a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88892c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd31a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50602/0 (socket says 172.21.15.92:50602) 2024-03-19T01:59:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88912d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 0x55ebbfd2c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd88892c640 1 -- 172.21.15.92:0/174373197 learned_addr learned my addr 172.21.15.92:0/174373197 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.966+0000 7fd87bfff640 1 --2- 172.21.15.92:0/174373197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd32070 0x55ebbfd34500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd879ffb640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd86c0a2920 0x7fd86c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd88892c640 1 -- 172.21.15.92:0/174373197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 msgr2=0x55ebbfd2c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd88892c640 1 --2- 172.21.15.92:0/174373197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 0x55ebbfd2c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd88892c640 1 -- 172.21.15.92:0/174373197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd32070 msgr2=0x55ebbfd34500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd88892c640 1 --2- 172.21.15.92:0/174373197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd32070 0x55ebbfd34500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd88892c640 1 -- 172.21.15.92:0/174373197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbfc2cd20 con 0x55ebbfd2cf00 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd88912d640 1 --2- 172.21.15.92:0/174373197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 0x55ebbfd2c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd88912d640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd86c0a2920 0x7fd86c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd88892c640 1 --2- 172.21.15.92:0/174373197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd31a00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd880009c80 tx=0x7fd880011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c610e520e625388c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd879ffb640 1 -- 172.21.15.92:0/3558253777 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd8740f0540 con 0x7fd884101930 2024-03-19T01:59:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd86affd640 1 -- 172.21.15.92:0/174373197 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88001e070 con 0x55ebbfd2cf00 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd86affd640 1 -- 172.21.15.92:0/174373197 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd880019b10 con 0x55ebbfd2cf00 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.967+0000 7fd86affd640 1 -- 172.21.15.92:0/174373197 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88000eae0 con 0x55ebbfd2cf00 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 -- 172.21.15.92:0/174373197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 msgr2=0x55ebbfd31a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88912d640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd86c0a2920 0x7fd86c0a4de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55ebbfd2d760 tx=0x7fd87c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 --2- 172.21.15.92:0/174373197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd31a00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd880009c80 tx=0x7fd880011810 comp rx=0 tx=0).stop 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 -- 172.21.15.92:0/174373197 shutdown_connections 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 --2- 172.21.15.92:0/174373197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd32070 0x55ebbfd34500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 --2- 172.21.15.92:0/174373197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd31a00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 --2- 172.21.15.92:0/174373197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 0x55ebbfd2c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 -- 172.21.15.92:0/174373197 >> 172.21.15.92:0/174373197 conn(0x55ebbfd23670 msgr2=0x55ebbfd23a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 -- 172.21.15.92:0/174373197 shutdown_connections 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 -- 172.21.15.92:0/174373197 wait complete. 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 -- start start 2024-03-19T01:59:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.968+0000 7fd88d780740 1 Processor -- start 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88d780740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 0x55ebbfd36510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88d780740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd36a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88d780740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd3da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88d780740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ebbfd35020 con 0x55ebbfd32070 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88d780740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ebbfc48db0 con 0x55ebbfd2c510 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88d780740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ebbfd351a0 con 0x55ebbfd2cf00 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88d780740 10 client.0 _ll_register_callbacks cb 0x55ebbfc4de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88912d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd3da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88912d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd3da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50606/0 (socket says 172.21.15.92:50606) 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88912d640 1 -- 172.21.15.92:0/4178482801 learned_addr learned my addr 172.21.15.92:0/4178482801 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd87bfff640 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd36a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88892c640 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 0x55ebbfd36510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88912d640 1 -- 172.21.15.92:0/4178482801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2cf00 msgr2=0x55ebbfd36a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88912d640 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd36a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88912d640 1 -- 172.21.15.92:0/4178482801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 msgr2=0x55ebbfd36510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88912d640 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 0x55ebbfd36510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88912d640 1 -- 172.21.15.92:0/4178482801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbfc47c60 con 0x55ebbfd32070 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd88912d640 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd3da90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd87c010580 tx=0x7fd87c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fbcd68d2eab1e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.969+0000 7fd863fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd863fff640 1 -- 172.21.15.92:0/4178482801 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c028640 con 0x55ebbfd32070 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd863fff640 1 -- 172.21.15.92:0/4178482801 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd87c01c400 con 0x55ebbfd32070 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88d780740 1 -- 172.21.15.92:0/4178482801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd880019b10 con 0x55ebbfd32070 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd863fff640 1 -- 172.21.15.92:0/4178482801 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd87c0287e0 con 0x55ebbfd32070 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88d780740 10 client.15852 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88d780740 1 -- 172.21.15.92:0/4178482801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ebbf9a1210 con 0x55ebbfd32070 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd863fff640 1 -- 172.21.15.92:0/4178482801 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd880019b10 con 0x55ebbfd32070 2024-03-19T01:59:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd863fff640 1 -- 172.21.15.92:0/4178482801 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd87c026070 con 0x55ebbfd32070 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88d780740 10 client.15852 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88d780740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88d780740 20 client.15852 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88d780740 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ebbfd42440 0x55ebbfd448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88d780740 4 client.15852 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88d780740 1 -- 172.21.15.92:0/4178482801 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebbfc9a120 con 0x55ebbfd42440 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.970+0000 7fd88892c640 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ebbfd42440 0x55ebbfd448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.971+0000 7fd88892c640 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ebbfd42440 0x55ebbfd448d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=849441bd30810b2b server_cookie=e44d8643aa211654 in_seq=0 out_seq=0 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.971+0000 7fd863fff640 10 client.15852 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.971+0000 7fd863fff640 1 -- 172.21.15.92:0/4178482801 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ebbfc9a120 con 0x55ebbfd42440 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.971+0000 7fd863fff640 10 client.15852 handle_command_reply: tid=0 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.971+0000 7fd88d780740 1 client.15852 shutdown 2024-03-19T01:59:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 -- 172.21.15.92:0/4178482801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 msgr2=0x55ebbfd3da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd3da90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd87c010580 tx=0x7fd87c004ab0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 -- 172.21.15.92:0/4178482801 shutdown_connections 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ebbfd42440 0x55ebbfd448d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd3da90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd863fff640 0 client.15852 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd36a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 --2- 172.21.15.92:0/4178482801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 0x55ebbfd36510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 -- 172.21.15.92:0/4178482801 >> 172.21.15.92:0/4178482801 conn(0x55ebbfd23670 msgr2=0x55ebbfd248c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 -- 172.21.15.92:0/4178482801 shutdown_connections 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 1 -- 172.21.15.92:0/4178482801 wait complete. 2024-03-19T01:59:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.972+0000 7fd88d780740 20 client.15852 trim_cache size 0 max 16384 2024-03-19T01:59:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.986+0000 7fd88d780740 1 Processor -- start 2024-03-19T01:59:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.986+0000 7fd88d780740 1 -- start start 2024-03-19T01:59:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.986+0000 7fd88d780740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 0x55ebbfd26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.986+0000 7fd88d780740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.986+0000 7fd88d780740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.986+0000 7fd88d780740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841a6aa0 con 0x55ebbfd32070 2024-03-19T01:59:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.986+0000 7fd88d780740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841a85e0 con 0x55ebbfd2cf00 2024-03-19T01:59:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.986+0000 7fd88d780740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841a7170 con 0x55ebbfd2c510 2024-03-19T01:59:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88912d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88912d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50622/0 (socket says 172.21.15.92:50622) 2024-03-19T01:59:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88892c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 0x55ebbfd26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88912d640 1 -- 172.21.15.92:0/243929707 learned_addr learned my addr 172.21.15.92:0/243929707 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd87bfff640 1 --2- 172.21.15.92:0/243929707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88912d640 1 -- 172.21.15.92:0/243929707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 msgr2=0x55ebbfd26fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88912d640 1 --2- 172.21.15.92:0/243929707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 0x55ebbfd26fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88912d640 1 -- 172.21.15.92:0/243929707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2cf00 msgr2=0x55ebbfd3da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88912d640 1 --2- 172.21.15.92:0/243929707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88912d640 1 -- 172.21.15.92:0/243929707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbfd1eee0 con 0x55ebbfd32070 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd88912d640 1 --2- 172.21.15.92:0/243929707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd491d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd87c004a10 tx=0x7fd87c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38ce4969fb67014 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd86a7fc640 1 -- 172.21.15.92:0/243929707 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c028640 con 0x55ebbfd32070 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd86a7fc640 1 -- 172.21.15.92:0/243929707 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87c02e030 con 0x55ebbfd32070 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.987+0000 7fd86a7fc640 1 -- 172.21.15.92:0/243929707 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c0323f0 con 0x55ebbfd32070 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 -- 172.21.15.92:0/243929707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 msgr2=0x55ebbfd491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 --2- 172.21.15.92:0/243929707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd491d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd87c004a10 tx=0x7fd87c009580 comp rx=0 tx=0).stop 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 -- 172.21.15.92:0/243929707 shutdown_connections 2024-03-19T01:59:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 --2- 172.21.15.92:0/243929707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd32070 0x55ebbfd491d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 --2- 172.21.15.92:0/243929707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 --2- 172.21.15.92:0/243929707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd2c510 0x55ebbfd26fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 -- 172.21.15.92:0/243929707 >> 172.21.15.92:0/243929707 conn(0x55ebbfd23670 msgr2=0x55ebbfd268d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 -- 172.21.15.92:0/243929707 shutdown_connections 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 -- 172.21.15.92:0/243929707 wait complete. 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 -- start start 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 Processor -- start 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 0x55ebbfd3bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd32070 0x55ebbfd3cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.988+0000 7fd88d780740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841a7f10 con 0x55ebbfd2cf00 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd88d780740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841a7840 con 0x55ebbfd2c510 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd88d780740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841a4a80 con 0x55ebbfd32070 2024-03-19T01:59:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd88d780740 10 client.0 _ll_register_callbacks cb 0x7fd87c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50630/0 (socket says 172.21.15.92:50630) 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd87bfff640 1 -- 172.21.15.92:0/2828328568 learned_addr learned my addr 172.21.15.92:0/2828328568 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd88892c640 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 0x55ebbfd3bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd88912d640 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd32070 0x55ebbfd3cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd87bfff640 1 -- 172.21.15.92:0/2828328568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd32070 msgr2=0x55ebbfd3cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd87bfff640 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd32070 0x55ebbfd3cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd87bfff640 1 -- 172.21.15.92:0/2828328568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 msgr2=0x55ebbfd3bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd87bfff640 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 0x55ebbfd3bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd87bfff640 1 -- 172.21.15.92:0/2828328568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbfa6d620 con 0x55ebbfd2cf00 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd87bfff640 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3c010 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd8740b9970 tx=0x7fd874005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb974c25c5b694a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd863fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd863fff640 1 -- 172.21.15.92:0/2828328568 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87404b040 con 0x55ebbfd2cf00 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd863fff640 1 -- 172.21.15.92:0/2828328568 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd8740f0110 con 0x55ebbfd2cf00 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd88d780740 1 -- 172.21.15.92:0/2828328568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd87c02e030 con 0x55ebbfd2cf00 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd863fff640 1 -- 172.21.15.92:0/2828328568 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8740bc900 con 0x55ebbfd2cf00 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd88d780740 10 client.15858 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd88d780740 1 -- 172.21.15.92:0/2828328568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ebbfda9010 con 0x55ebbfd2cf00 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.989+0000 7fd863fff640 1 -- 172.21.15.92:0/2828328568 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87c02e030 con 0x55ebbfd2cf00 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd863fff640 1 -- 172.21.15.92:0/2828328568 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd8740c5200 con 0x55ebbfd2cf00 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd88d780740 10 client.15858 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd88d780740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd88d780740 20 client.15858 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd88d780740 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ebbfda9a10 0x55ebbfdabe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd88d780740 4 client.15858 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:59:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd88d780740 1 -- 172.21.15.92:0/2828328568 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ebbfc94470 con 0x55ebbfda9a10 2024-03-19T01:59:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd88892c640 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ebbfda9a10 0x55ebbfdabe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd863fff640 10 client.15858 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.990+0000 7fd88892c640 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ebbfda9a10 0x55ebbfdabe30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f47eff43aa29df server_cookie=e6a09fb35c2a8171 in_seq=0 out_seq=0 2024-03-19T01:59:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd863fff640 1 -- 172.21.15.92:0/2828328568 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ebbfc94470 con 0x55ebbfda9a10 2024-03-19T01:59:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd863fff640 10 client.15858 handle_command_reply: tid=0 2024-03-19T01:59:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 client.15858 shutdown 2024-03-19T01:59:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 -- 172.21.15.92:0/2828328568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 msgr2=0x55ebbfd3c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3c010 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd8740b9970 tx=0x7fd874005c90 comp rx=0 tx=0).stop 2024-03-19T01:59:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 -- 172.21.15.92:0/2828328568 shutdown_connections 2024-03-19T01:59:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ebbfda9a10 0x55ebbfdabe30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebbfd32070 0x55ebbfd3cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ebbfd2cf00 0x55ebbfd3c010 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd863fff640 0 client.15858 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 --2- 172.21.15.92:0/2828328568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebbfd2c510 0x55ebbfd3bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 -- 172.21.15.92:0/2828328568 >> 172.21.15.92:0/2828328568 conn(0x55ebbfd23670 msgr2=0x55ebbfd27160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 -- 172.21.15.92:0/2828328568 shutdown_connections 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 1 -- 172.21.15.92:0/2828328568 wait complete. 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.997+0000 7fd88d780740 20 client.15858 trim_cache size 0 max 16384 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:59:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:59:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 -- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd86c0a2920 msgr2=0x7fd86c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd86c0a2920 0x7fd86c0a4de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55ebbfd2d760 tx=0x7fd87c004a40 comp rx=0 tx=0).stop 2024-03-19T01:59:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 -- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 msgr2=0x7fd884197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 0x7fd884197ff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd874009230 tx=0x7fd87402c650 comp rx=0 tx=0).stop 2024-03-19T01:59:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 -- 172.21.15.92:0/3558253777 shutdown_connections 2024-03-19T01:59:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8841022f0 0x7fd88419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd884101930 0x7fd884197ff0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd86c0a2920 0x7fd86c0a4de0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 --2- 172.21.15.92:0/3558253777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884100740 0x7fd884197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 -- 172.21.15.92:0/3558253777 >> 172.21.15.92:0/3558253777 conn(0x7fd8840fbcb0 msgr2=0x7fd8840fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 -- 172.21.15.92:0/3558253777 shutdown_connections 2024-03-19T01:59:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:28.998+0000 7fd88ab90640 1 -- 172.21.15.92:0/3558253777 wait complete. 2024-03-19T01:59:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:28 smithi092 ceph-mon[62574]: pgmap v551: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:59:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:28 smithi144 ceph-mon[67199]: pgmap v551: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T01:59:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:30 smithi159 ceph-mon[67110]: pgmap v552: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:59:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:30 smithi092 ceph-mon[62574]: pgmap v552: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:59:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:30 smithi144 ceph-mon[67199]: pgmap v552: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 19 op/s 2024-03-19T01:59:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:32 smithi159 ceph-mon[67110]: pgmap v553: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-03-19T01:59:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:32 smithi092 ceph-mon[62574]: pgmap v553: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-03-19T01:59:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:32 smithi144 ceph-mon[67199]: pgmap v553: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-03-19T01:59:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:34 smithi159 ceph-mon[67110]: pgmap v554: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:34 smithi092 ceph-mon[62574]: pgmap v554: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:34 smithi144 ceph-mon[67199]: pgmap v554: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:36 smithi159 ceph-mon[67110]: pgmap v555: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:36 smithi092 ceph-mon[62574]: pgmap v555: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:36 smithi144 ceph-mon[67199]: pgmap v555: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:38 smithi159 ceph-mon[67110]: pgmap v556: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:38 smithi092 ceph-mon[62574]: pgmap v556: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:38 smithi144 ceph-mon[67199]: pgmap v556: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:40 smithi159 ceph-mon[67110]: pgmap v557: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:59:41.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:40 smithi092 ceph-mon[62574]: pgmap v557: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:59:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:40 smithi144 ceph-mon[67199]: pgmap v557: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:59:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.086+0000 7f16763b9640 1 -- 172.21.15.92:0/181810011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100a90 msgr2=0x7f1670100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.086+0000 7f16763b9640 1 --2- 172.21.15.92:0/181810011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100a90 0x7f1670100ef0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1664009900 tx=0x7f166402bc30 comp rx=0 tx=0).stop 2024-03-19T01:59:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.087+0000 7f16763b9640 1 -- 172.21.15.92:0/181810011 shutdown_connections 2024-03-19T01:59:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.087+0000 7f16763b9640 1 --2- 172.21.15.92:0/181810011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1670101430 0x7f16701018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.087+0000 7f16763b9640 1 --2- 172.21.15.92:0/181810011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100a90 0x7f1670100ef0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.087+0000 7f16763b9640 1 --2- 172.21.15.92:0/181810011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1670100130 0x7f1670100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.087+0000 7f16763b9640 1 -- 172.21.15.92:0/181810011 >> 172.21.15.92:0/181810011 conn(0x7f16700fbcd0 msgr2=0x7f16700fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.087+0000 7f16763b9640 1 -- 172.21.15.92:0/181810011 shutdown_connections 2024-03-19T01:59:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.087+0000 7f16763b9640 1 -- 172.21.15.92:0/181810011 wait complete. 2024-03-19T01:59:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.087+0000 7f16763b9640 1 Processor -- start 2024-03-19T01:59:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.087+0000 7f16763b9640 1 -- start start 2024-03-19T01:59:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f16763b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100130 0x7f1670199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f16763b9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1670100a90 0x7f167019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f16763b9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1670101430 0x7f16701a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f16763b9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f167010f6d0 con 0x7f1670100130 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f16763b9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f167010f550 con 0x7f1670101430 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f16763b9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f167010f850 con 0x7f1670100a90 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100130 0x7f1670199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100130 0x7f1670199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59736/0 (socket says 172.21.15.92:59736) 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166ffff640 1 -- 172.21.15.92:0/3941931244 learned_addr learned my addr 172.21.15.92:0/3941931244 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166f7fe640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1670100a90 0x7f167019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f1674956640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1670101430 0x7f16701a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166ffff640 1 -- 172.21.15.92:0/3941931244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1670100a90 msgr2=0x7f167019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166ffff640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1670100a90 0x7f167019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166ffff640 1 -- 172.21.15.92:0/3941931244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1670101430 msgr2=0x7f16701a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166ffff640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1670101430 0x7f16701a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166ffff640 1 -- 172.21.15.92:0/3941931244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16701a1900 con 0x7f1670100130 2024-03-19T01:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.088+0000 7f166ffff640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100130 0x7f1670199d10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f166000ea60 tx=0x7f166000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.089+0000 7f166d7fa640 1 -- 172.21.15.92:0/3941931244 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000cde0 con 0x7f1670100130 2024-03-19T01:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.089+0000 7f16763b9640 1 -- 172.21.15.92:0/3941931244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16701a1bf0 con 0x7f1670100130 2024-03-19T01:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.089+0000 7f166d7fa640 1 -- 172.21.15.92:0/3941931244 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16600035f0 con 0x7f1670100130 2024-03-19T01:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.089+0000 7f166d7fa640 1 -- 172.21.15.92:0/3941931244 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660010670 con 0x7f1670100130 2024-03-19T01:59:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.089+0000 7f16763b9640 1 -- 172.21.15.92:0/3941931244 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16701a2110 con 0x7f1670100130 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.091+0000 7f1678fa6740 1 Processor -- start 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f166d7fa640 1 -- 172.21.15.92:0/3941931244 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f16600037a0 con 0x7f1670100130 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1678fa6740 1 -- start start 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9e100 0x563136a9e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aa35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aa60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1678fa6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563136a02be0 con 0x563136a9e100 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1678fa6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5631369ab960 con 0x563136a9eaf0 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1678fa6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5631369fec80 con 0x563136aa3c60 2024-03-19T01:59:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1674956640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9e100 0x563136a9e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1674956640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9e100 0x563136a9e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59752/0 (socket says 172.21.15.92:59752) 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1674956640 1 -- 172.21.15.92:0/3568096180 learned_addr learned my addr 172.21.15.92:0/3568096180 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f166ffff640 1 --2- 172.21.15.92:0/3568096180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aa35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f166f7fe640 1 --2- 172.21.15.92:0/3568096180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aa60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1674956640 1 -- 172.21.15.92:0/3568096180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 msgr2=0x563136aa60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1674956640 1 --2- 172.21.15.92:0/3568096180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aa60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1674956640 1 -- 172.21.15.92:0/3568096180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 msgr2=0x563136aa35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1674956640 1 --2- 172.21.15.92:0/3568096180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aa35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1674956640 1 -- 172.21.15.92:0/3568096180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631369a2d20 con 0x563136a9e100 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f166d7fa640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16580a28a0 0x7f16580a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f166ffff640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16580a28a0 0x7f16580a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f1674956640 1 --2- 172.21.15.92:0/3568096180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9e100 0x563136a9e520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1668009fd0 tx=0x7f16680118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cc8a35afdba4a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f166d7fa640 1 -- 172.21.15.92:0/3941931244 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f16600ced30 con 0x7f1670100130 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.092+0000 7f16567fc640 1 -- 172.21.15.92:0/3568096180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668004d20 con 0x563136a9e100 2024-03-19T01:59:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f16567fc640 1 -- 172.21.15.92:0/3568096180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1668004050 con 0x563136a9e100 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f16567fc640 1 -- 172.21.15.92:0/3568096180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668005280 con 0x563136a9e100 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f1678fa6740 1 -- 172.21.15.92:0/3568096180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9e100 msgr2=0x563136a9e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f166ffff640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16580a28a0 0x7f16580a4d60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f16600ccbb0 tx=0x7f16600d2810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f1678fa6740 1 --2- 172.21.15.92:0/3568096180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9e100 0x563136a9e520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1668009fd0 tx=0x7f16680118d0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f1678fa6740 1 -- 172.21.15.92:0/3568096180 shutdown_connections 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f1678fa6740 1 --2- 172.21.15.92:0/3568096180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aa60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f1678fa6740 1 --2- 172.21.15.92:0/3568096180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aa35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f1678fa6740 1 --2- 172.21.15.92:0/3568096180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9e100 0x563136a9e520 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f1678fa6740 1 -- 172.21.15.92:0/3568096180 >> 172.21.15.92:0/3568096180 conn(0x563136a99670 msgr2=0x563136a99a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f1678fa6740 1 -- 172.21.15.92:0/3568096180 shutdown_connections 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.093+0000 7f1678fa6740 1 -- 172.21.15.92:0/3568096180 wait complete. 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1678fa6740 1 -- start start 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1678fa6740 1 Processor -- start 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 0x563136aa8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136aa8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1678fa6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563136aa6c10 con 0x563136a9eaf0 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1678fa6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5631369bedb0 con 0x563136a9e100 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1678fa6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563136aa6d90 con 0x563136aa3c60 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1678fa6740 10 client.0 _ll_register_callbacks cb 0x5631369c3e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136aa8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136aa8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59766/0 (socket says 172.21.15.92:59766) 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f166f7fe640 1 -- 172.21.15.92:0/2578175730 learned_addr learned my addr 172.21.15.92:0/2578175730 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1674956640 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1674956640 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 0x563136aa8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f166f7fe640 1 -- 172.21.15.92:0/2578175730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 msgr2=0x563136aaf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f166f7fe640 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f166f7fe640 1 -- 172.21.15.92:0/2578175730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 msgr2=0x563136aa8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f166f7fe640 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 0x563136aa8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f166f7fe640 1 -- 172.21.15.92:0/2578175730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631369bdc60 con 0x563136a9eaf0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.094+0000 7f1674956640 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f166f7fe640 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136aa8590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1664009900 tx=0x7f1664004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482441c46823e0e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f164f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f164f7fe640 1 -- 172.21.15.92:0/2578175730 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664005d50 con 0x563136a9eaf0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f164f7fe640 1 -- 172.21.15.92:0/2578175730 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1664005230 con 0x563136a9eaf0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f1678fa6740 1 -- 172.21.15.92:0/2578175730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1668004050 con 0x563136a9eaf0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f164f7fe640 1 -- 172.21.15.92:0/2578175730 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1664005430 con 0x563136a9eaf0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f1678fa6740 10 client.15870 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f1678fa6740 1 -- 172.21.15.92:0/2578175730 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563136717210 con 0x563136a9eaf0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f164f7fe640 1 -- 172.21.15.92:0/2578175730 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1668004050 con 0x563136a9eaf0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f164f7fe640 1 -- 172.21.15.92:0/2578175730 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f16640037b0 con 0x563136a9eaf0 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f1678fa6740 10 client.15870 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f1678fa6740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f1678fa6740 20 client.15870 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563136ab3ee0 0x563136ab6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f1678fa6740 4 client.15870 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.095+0000 7f1678fa6740 1 -- 172.21.15.92:0/2578175730 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563136a10120 con 0x563136ab3ee0 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.096+0000 7f1674956640 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563136ab3ee0 0x563136ab6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.096+0000 7f164f7fe640 10 client.15870 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.096+0000 7f1674956640 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563136ab3ee0 0x563136ab6360 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ce7d47d7cc00414 server_cookie=f49e423bb1a7a666 in_seq=0 out_seq=0 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.096+0000 7f164f7fe640 1 -- 172.21.15.92:0/2578175730 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563136a10120 con 0x563136ab3ee0 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.096+0000 7f164f7fe640 10 client.15870 handle_command_reply: tid=0 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 client.15870 shutdown 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 -- 172.21.15.92:0/2578175730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 msgr2=0x563136aa8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136aa8590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1664009900 tx=0x7f1664004270 comp rx=0 tx=0).stop 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 -- 172.21.15.92:0/2578175730 shutdown_connections 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563136ab3ee0 0x563136ab6360 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136aa8590 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f164f7fe640 0 client.15870 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2578175730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 0x563136aa8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 -- 172.21.15.92:0/2578175730 >> 172.21.15.92:0/2578175730 conn(0x563136a99670 msgr2=0x563136a9a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 -- 172.21.15.92:0/2578175730 shutdown_connections 2024-03-19T01:59:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 1 -- 172.21.15.92:0/2578175730 wait complete. 2024-03-19T01:59:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.097+0000 7f1678fa6740 20 client.15870 trim_cache size 0 max 16384 2024-03-19T01:59:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.111+0000 7f1678fa6740 1 Processor -- start 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.111+0000 7f1678fa6740 1 -- start start 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 0x563136a98220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136a98760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1678fa6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16701aae30 con 0x563136a9eaf0 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1678fa6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ac970 con 0x563136a9e100 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1678fa6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ab500 con 0x563136aa3c60 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136a98760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136a98760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59774/0 (socket says 172.21.15.92:59774) 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f166f7fe640 1 -- 172.21.15.92:0/2075101947 learned_addr learned my addr 172.21.15.92:0/2075101947 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1674956640 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 0x563136a98220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1674956640 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f166f7fe640 1 -- 172.21.15.92:0/2075101947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 msgr2=0x563136aaf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f166f7fe640 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f166f7fe640 1 -- 172.21.15.92:0/2075101947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 msgr2=0x563136a98220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f166f7fe640 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 0x563136a98220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f166f7fe640 1 -- 172.21.15.92:0/2075101947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631367e3620 con 0x563136a9eaf0 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1674956640 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f166f7fe640 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136a98760 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1664009850 tx=0x7f166402cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc93778a98d0119 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1655ffb640 1 -- 172.21.15.92:0/2075101947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664009520 con 0x563136a9eaf0 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1655ffb640 1 -- 172.21.15.92:0/2075101947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1664046480 con 0x563136a9eaf0 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.112+0000 7f1655ffb640 1 -- 172.21.15.92:0/2075101947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664046c00 con 0x563136a9eaf0 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 -- 172.21.15.92:0/2075101947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 msgr2=0x563136a98760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136a98760 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1664009850 tx=0x7f166402cec0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 -- 172.21.15.92:0/2075101947 shutdown_connections 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136aa3c60 0x563136aaf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136a9eaf0 0x563136a98760 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 --2- 172.21.15.92:0/2075101947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9e100 0x563136a98220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 -- 172.21.15.92:0/2075101947 >> 172.21.15.92:0/2075101947 conn(0x563136a99670 msgr2=0x563136a99a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 -- 172.21.15.92:0/2075101947 shutdown_connections 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 -- 172.21.15.92:0/2075101947 wait complete. 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 -- start start 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.113+0000 7f1678fa6740 1 Processor -- start 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136a9e100 0x563136aacb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1678fa6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136aa3c60 0x563136aae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1678fa6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ac2a0 con 0x563136aa3c60 2024-03-19T01:59:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1678fa6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16701abbd0 con 0x563136a9eaf0 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1678fa6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16701a8e10 con 0x563136a9e100 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1678fa6740 10 client.0 _ll_register_callbacks cb 0x7f1664009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1674956640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136aa3c60 0x563136aae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:60730/0 (socket says 172.21.15.92:60730) 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f166f7fe640 1 -- 172.21.15.92:0/1443111711 learned_addr learned my addr 172.21.15.92:0/1443111711 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1674956640 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136a9e100 0x563136aacb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f166f7fe640 1 -- 172.21.15.92:0/1443111711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136a9e100 msgr2=0x563136aacb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f166f7fe640 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136a9e100 0x563136aacb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f166f7fe640 1 -- 172.21.15.92:0/1443111711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136aa3c60 msgr2=0x563136aae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f166f7fe640 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136aa3c60 0x563136aae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f166f7fe640 1 -- 172.21.15.92:0/1443111711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563136aae690 con 0x563136a9eaf0 2024-03-19T01:59:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1674956640 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136aa3c60 0x563136aae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f166f7fe640 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aad090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1664003c50 tx=0x7f16640038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21313ee9cb981e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f164f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f164f7fe640 1 -- 172.21.15.92:0/1443111711 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664055070 con 0x563136a9eaf0 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f164f7fe640 1 -- 172.21.15.92:0/1443111711 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1664046b90 con 0x563136a9eaf0 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.114+0000 7f1678fa6740 1 -- 172.21.15.92:0/1443111711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1664046480 con 0x563136a9eaf0 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f164f7fe640 1 -- 172.21.15.92:0/1443111711 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1664009520 con 0x563136a9eaf0 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f1678fa6740 10 client.25709 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f1678fa6740 1 -- 172.21.15.92:0/1443111711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563136b1a260 con 0x563136a9eaf0 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f164f7fe640 1 -- 172.21.15.92:0/1443111711 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1664046480 con 0x563136a9eaf0 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f164f7fe640 1 -- 172.21.15.92:0/1443111711 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1664051bd0 con 0x563136a9eaf0 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f1678fa6740 10 client.25709 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f1678fa6740 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f1678fa6740 20 client.25709 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f1678fa6740 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563136b1aa30 0x563136b1ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f1678fa6740 4 client.25709 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:59:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f1678fa6740 1 -- 172.21.15.92:0/1443111711 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563136a0a470 con 0x563136b1aa30 2024-03-19T01:59:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.115+0000 7f1674956640 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563136b1aa30 0x563136b1ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.116+0000 7f164f7fe640 10 client.25709 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.116+0000 7f1674956640 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563136b1aa30 0x563136b1ce50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7599351f647a3a7a server_cookie=f0970e3757f44b77 in_seq=0 out_seq=0 2024-03-19T01:59:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f164f7fe640 1 -- 172.21.15.92:0/1443111711 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563136a0a470 con 0x563136b1aa30 2024-03-19T01:59:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f164f7fe640 10 client.25709 handle_command_reply: tid=0 2024-03-19T01:59:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 client.25709 shutdown 2024-03-19T01:59:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 -- 172.21.15.92:0/1443111711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 msgr2=0x563136aad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aad090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1664003c50 tx=0x7f16640038a0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 -- 172.21.15.92:0/1443111711 shutdown_connections 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x563136b1aa30 0x563136b1ce50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563136aa3c60 0x563136aae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563136a9eaf0 0x563136aad090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f164f7fe640 0 client.25709 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 --2- 172.21.15.92:0/1443111711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563136a9e100 0x563136aacb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 -- 172.21.15.92:0/1443111711 >> 172.21.15.92:0/1443111711 conn(0x563136a99670 msgr2=0x563136a98220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 -- 172.21.15.92:0/1443111711 shutdown_connections 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 1 -- 172.21.15.92:0/1443111711 wait complete. 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.122+0000 7f1678fa6740 20 client.25709 trim_cache size 0 max 16384 2024-03-19T01:59:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:59:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:59:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:59:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:59:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 -- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16580a28a0 msgr2=0x7f16580a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16580a28a0 0x7f16580a4d60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f16600ccbb0 tx=0x7f16600d2810 comp rx=0 tx=0).stop 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 -- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100130 msgr2=0x7f1670199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100130 0x7f1670199d10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f166000ea60 tx=0x7f166000ef20 comp rx=0 tx=0).stop 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 -- 172.21.15.92:0/3941931244 shutdown_connections 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1670101430 0x7f16701a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f16580a28a0 0x7f16580a4d60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1670100a90 0x7f167019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 --2- 172.21.15.92:0/3941931244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1670100130 0x7f1670199d10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 -- 172.21.15.92:0/3941931244 >> 172.21.15.92:0/3941931244 conn(0x7f16700fbcd0 msgr2=0x7f16700fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 -- 172.21.15.92:0/3941931244 shutdown_connections 2024-03-19T01:59:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:42.123+0000 7f16763b9640 1 -- 172.21.15.92:0/3941931244 wait complete. 2024-03-19T01:59:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:42 smithi159 ceph-mon[67110]: pgmap v558: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T01:59:43.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:42 smithi092 ceph-mon[62574]: pgmap v558: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T01:59:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:42 smithi144 ceph-mon[67199]: pgmap v558: 97 pgs: 97 active+clean; 2.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T01:59:44.957 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:44 smithi159 ceph-mon[67110]: pgmap v559: 97 pgs: 97 active+clean; 2.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:44 smithi092 ceph-mon[62574]: pgmap v559: 97 pgs: 97 active+clean; 2.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:44 smithi144 ceph-mon[67199]: pgmap v559: 97 pgs: 97 active+clean; 2.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:46 smithi092 ceph-mon[62574]: pgmap v560: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-03-19T01:59:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:46 smithi144 ceph-mon[67199]: pgmap v560: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-03-19T01:59:47.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:46 smithi159 ceph-mon[67110]: pgmap v560: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-03-19T01:59:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:48 smithi092 ceph-mon[62574]: pgmap v561: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:59:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:48 smithi144 ceph-mon[67199]: pgmap v561: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:59:49.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:48 smithi159 ceph-mon[67110]: pgmap v561: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T01:59:51.007 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:50 smithi159 ceph-mon[67110]: pgmap v562: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:59:51.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:50 smithi092 ceph-mon[62574]: pgmap v562: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:59:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:50 smithi144 ceph-mon[67199]: pgmap v562: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T01:59:53.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:52 smithi092 ceph-mon[62574]: pgmap v563: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:52 smithi144 ceph-mon[67199]: pgmap v563: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:53.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:52 smithi159 ceph-mon[67110]: pgmap v563: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T01:59:55.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:54 smithi092 ceph-mon[62574]: pgmap v564: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:55.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:54 smithi144 ceph-mon[67199]: pgmap v564: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:55.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:54 smithi159 ceph-mon[67110]: pgmap v564: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T01:59:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.209+0000 7ff608d9d640 1 -- 172.21.15.92:0/1842921371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6041020f0 msgr2=0x7ff6041044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.209+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1842921371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6041020f0 0x7ff6041044e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009ca0 tx=0x7ff5f402c0f0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.209+0000 7ff608d9d640 1 -- 172.21.15.92:0/1842921371 shutdown_connections 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1842921371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff604107350 0x7ff6041097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1842921371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604104a20 0x7ff604106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1842921371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6041020f0 0x7ff6041044e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 -- 172.21.15.92:0/1842921371 >> 172.21.15.92:0/1842921371 conn(0x7ff6040fbb00 msgr2=0x7ff6040fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 -- 172.21.15.92:0/1842921371 shutdown_connections 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 -- 172.21.15.92:0/1842921371 wait complete. 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 Processor -- start 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 -- start start 2024-03-19T01:59:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6041020f0 0x7ff604197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff604104a20 0x7ff604197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604107350 0x7ff60419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410d310 con 0x7ff604104a20 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410d190 con 0x7ff604107350 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.210+0000 7ff608d9d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410d490 con 0x7ff6041020f0 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff602ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff604104a20 0x7ff604197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6041020f0 0x7ff604197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff602ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff604104a20 0x7ff604197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42774/0 (socket says 172.21.15.92:42774) 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff602ffd640 1 -- 172.21.15.92:0/1677046082 learned_addr learned my addr 172.21.15.92:0/1677046082 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff603fff640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604107350 0x7ff60419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff602ffd640 1 -- 172.21.15.92:0/1677046082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6041020f0 msgr2=0x7ff604197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff602ffd640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6041020f0 0x7ff604197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff602ffd640 1 -- 172.21.15.92:0/1677046082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604107350 msgr2=0x7ff60419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff602ffd640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604107350 0x7ff60419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff602ffd640 1 -- 172.21.15.92:0/1677046082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60419f5a0 con 0x7ff604104a20 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff6037fe640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6041020f0 0x7ff604197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff603fff640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604107350 0x7ff60419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff602ffd640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff604104a20 0x7ff604197ec0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00e9c0 tx=0x7ff5ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff600ff9640 1 -- 172.21.15.92:0/1677046082 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00cd80 con 0x7ff604104a20 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff608d9d640 1 -- 172.21.15.92:0/1677046082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60419f890 con 0x7ff604104a20 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff600ff9640 1 -- 172.21.15.92:0/1677046082 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec005500 con 0x7ff604104a20 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff600ff9640 1 -- 172.21.15.92:0/1677046082 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec010690 con 0x7ff604104a20 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.211+0000 7ff608d9d640 1 -- 172.21.15.92:0/1677046082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60419fe30 con 0x7ff604104a20 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.213+0000 7ff600ff9640 1 -- 172.21.15.92:0/1677046082 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff5ec0046c0 con 0x7ff604104a20 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.214+0000 7ff600ff9640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5f00a27f0 0x7ff5f00a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.214+0000 7ff6037fe640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5f00a27f0 0x7ff5f00a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.214+0000 7ff600ff9640 1 -- 172.21.15.92:0/1677046082 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff5ec0cedb0 con 0x7ff604104a20 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.214+0000 7ff6037fe640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5f00a27f0 0x7ff5f00a4cb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4006010 tx=0x7ff5f4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff60cbe1740 1 Processor -- start 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff60cbe1740 1 -- start start 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47df700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47e4e40 0x55e2a47e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e2a4743be0 con 0x55e2a47df2e0 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e2a46ec960 con 0x55e2a47e4e40 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e2a473fc80 con 0x55e2a47dfcd0 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47df700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42790/0 (socket says 172.21.15.92:42790) 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff6037fe640 1 -- 172.21.15.92:0/1513241779 learned_addr learned my addr 172.21.15.92:0/1513241779 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff602ffd640 1 --2- 172.21.15.92:0/1513241779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff603fff640 1 --2- 172.21.15.92:0/1513241779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47e4e40 0x55e2a47e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff6037fe640 1 -- 172.21.15.92:0/1513241779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 msgr2=0x55e2a47e47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff6037fe640 1 --2- 172.21.15.92:0/1513241779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff6037fe640 1 -- 172.21.15.92:0/1513241779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47e4e40 msgr2=0x55e2a47e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff6037fe640 1 --2- 172.21.15.92:0/1513241779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47e4e40 0x55e2a47e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff6037fe640 1 -- 172.21.15.92:0/1513241779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a46e3d20 con 0x55e2a47df2e0 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.215+0000 7ff6037fe640 1 --2- 172.21.15.92:0/1513241779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47df700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009ca0 tx=0x7ff5f403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f979049b0aff5c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff5f9ffb640 1 -- 172.21.15.92:0/1513241779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4050070 con 0x55e2a47df2e0 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff5f9ffb640 1 -- 172.21.15.92:0/1513241779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f4003e50 con 0x55e2a47df2e0 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff5f9ffb640 1 -- 172.21.15.92:0/1513241779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4044070 con 0x55e2a47df2e0 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1513241779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 msgr2=0x55e2a47df700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1513241779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47df700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009ca0 tx=0x7ff5f403e000 comp rx=0 tx=0).stop 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1513241779 shutdown_connections 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1513241779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47e4e40 0x55e2a47e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1513241779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1513241779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47df700 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1513241779 >> 172.21.15.92:0/1513241779 conn(0x55e2a47da850 msgr2=0x55e2a47dac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1513241779 shutdown_connections 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.216+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1513241779 wait complete. 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff60cbe1740 1 -- start start 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff60cbe1740 1 Processor -- start 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 0x55e2a47e92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47e4e40 0x55e2a47f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e2a47e7d90 con 0x55e2a47e4e40 2024-03-19T01:59:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e2a46ffdb0 con 0x55e2a47dfcd0 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e2a47e7f10 con 0x55e2a47df2e0 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff60cbe1740 10 client.0 _ll_register_callbacks cb 0x55e2a4704e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff603fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47e4e40 0x55e2a47f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 0x55e2a47e92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff602ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff603fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47e4e40 0x55e2a47f07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42796/0 (socket says 172.21.15.92:42796) 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff602ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:36624/0 (socket says 172.21.15.92:36624) 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff603fff640 1 -- 172.21.15.92:0/3298974262 learned_addr learned my addr 172.21.15.92:0/3298974262 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff603fff640 1 -- 172.21.15.92:0/3298974262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 msgr2=0x55e2a47e92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff603fff640 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 0x55e2a47e92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff603fff640 1 -- 172.21.15.92:0/3298974262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 msgr2=0x55e2a47e9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff603fff640 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff603fff640 1 -- 172.21.15.92:0/3298974262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a46fec60 con 0x55e2a47e4e40 2024-03-19T01:59:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.217+0000 7ff602ffd640 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff603fff640 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47e4e40 0x55e2a47f07f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc012d20 tx=0x7ff5fc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5decc4c351abe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff5e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/3298974262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc009e30 con 0x55e2a47e4e40 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/3298974262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff5fc012110 con 0x55e2a47e4e40 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3298974262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f4003e50 con 0x55e2a47e4e40 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/3298974262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5fc004050 con 0x55e2a47e4e40 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff60cbe1740 10 client.15888 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3298974262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e2a4458210 con 0x55e2a47e4e40 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/3298974262 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f4003e50 con 0x55e2a47e4e40 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/3298974262 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff5fc010430 con 0x55e2a47e4e40 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff60cbe1740 10 client.15888 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff60cbe1740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff60cbe1740 20 client.15888 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e2a47f51a0 0x55e2a47f7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff60cbe1740 4 client.15888 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T01:59:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.218+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3298974262 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a4751120 con 0x55e2a47f51a0 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.219+0000 7ff6037fe640 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e2a47f51a0 0x55e2a47f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.219+0000 7ff5e2ffd640 10 client.15888 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.219+0000 7ff6037fe640 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e2a47f51a0 0x55e2a47f7590 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7077ada3c0b2230 server_cookie=71019539f9d38ced in_seq=0 out_seq=0 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/3298974262 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e2a4751120 con 0x55e2a47f51a0 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff5e2ffd640 10 client.15888 handle_command_reply: tid=0 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 client.15888 shutdown 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3298974262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47e4e40 msgr2=0x55e2a47f07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47e4e40 0x55e2a47f07f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc012d20 tx=0x7ff5fc00f790 comp rx=0 tx=0).stop 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3298974262 shutdown_connections 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e2a47f51a0 0x55e2a47f7590 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47e4e40 0x55e2a47f07f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47e9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff5e2ffd640 0 client.15888 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3298974262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 0x55e2a47e92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3298974262 >> 172.21.15.92:0/3298974262 conn(0x55e2a47da850 msgr2=0x55e2a47d9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3298974262 shutdown_connections 2024-03-19T01:59:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3298974262 wait complete. 2024-03-19T01:59:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.220+0000 7ff60cbe1740 20 client.15888 trim_cache size 0 max 16384 2024-03-19T01:59:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.234+0000 7ff60cbe1740 1 Processor -- start 2024-03-19T01:59:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.234+0000 7ff60cbe1740 1 -- start start 2024-03-19T01:59:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47da850 0x55e2a47d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 0x55e2a47fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041a8b70 con 0x55e2a47df2e0 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041aa6b0 con 0x55e2a47da850 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041a9240 con 0x55e2a47dfcd0 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff602ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff602ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47f07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42806/0 (socket says 172.21.15.92:42806) 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff602ffd640 1 -- 172.21.15.92:0/3903163479 learned_addr learned my addr 172.21.15.92:0/3903163479 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff6037fe640 1 --2- 172.21.15.92:0/3903163479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47da850 0x55e2a47d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff603fff640 1 --2- 172.21.15.92:0/3903163479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 0x55e2a47fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff602ffd640 1 -- 172.21.15.92:0/3903163479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 msgr2=0x55e2a47fbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff602ffd640 1 --2- 172.21.15.92:0/3903163479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 0x55e2a47fbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff602ffd640 1 -- 172.21.15.92:0/3903163479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47da850 msgr2=0x55e2a47d9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff602ffd640 1 --2- 172.21.15.92:0/3903163479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47da850 0x55e2a47d9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff602ffd640 1 -- 172.21.15.92:0/3903163479 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a4524620 con 0x55e2a47df2e0 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff602ffd640 1 --2- 172.21.15.92:0/3903163479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47f07b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec09b6f0 tx=0x7ff5ec086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2617d5ff35660315 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff5f97fa640 1 -- 172.21.15.92:0/3903163479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec09b760 con 0x55e2a47df2e0 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff5f97fa640 1 -- 172.21.15.92:0/3903163479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec09f8b0 con 0x55e2a47df2e0 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.235+0000 7ff5f97fa640 1 -- 172.21.15.92:0/3903163479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0a3a70 con 0x55e2a47df2e0 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3903163479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 msgr2=0x55e2a47f07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3903163479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47f07b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec09b6f0 tx=0x7ff5ec086040 comp rx=0 tx=0).stop 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3903163479 shutdown_connections 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3903163479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47dfcd0 0x55e2a47fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3903163479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47df2e0 0x55e2a47f07b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/3903163479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47da850 0x55e2a47d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3903163479 >> 172.21.15.92:0/3903163479 conn(0x55e2a47e8ea0 msgr2=0x55e2a47d9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3903163479 shutdown_connections 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 -- 172.21.15.92:0/3903163479 wait complete. 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.236+0000 7ff60cbe1740 1 -- start start 2024-03-19T01:59:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff60cbe1740 1 Processor -- start 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47da850 0x55e2a47ee180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 0x55e2a47ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff60cbe1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041a9fe0 con 0x55e2a47da850 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041a9910 con 0x55e2a47dfcd0 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff60cbe1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff6041a6b50 con 0x55e2a47df2e0 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff60cbe1740 10 client.0 _ll_register_callbacks cb 0x7ff5fc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47da850 0x55e2a47ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47da850 0x55e2a47ee180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42822/0 (socket says 172.21.15.92:42822) 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff6037fe640 1 -- 172.21.15.92:0/1804840938 learned_addr learned my addr 172.21.15.92:0/1804840938 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff603fff640 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff602ffd640 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 0x55e2a47ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff6037fe640 1 -- 172.21.15.92:0/1804840938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 msgr2=0x55e2a47ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff6037fe640 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 0x55e2a47ee6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff6037fe640 1 -- 172.21.15.92:0/1804840938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 msgr2=0x55e2a47ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff6037fe640 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff6037fe640 1 -- 172.21.15.92:0/1804840938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a47efcc0 con 0x55e2a47da850 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff603fff640 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff6037fe640 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47da850 0x55e2a47ee180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff5f402cdd0 tx=0x7ff5f4032a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f2a2928f575caa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.237+0000 7ff5e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/1804840938 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4044040 con 0x55e2a47da850 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/1804840938 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff5f4034ad0 con 0x55e2a47da850 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1804840938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5ec09f8b0 con 0x55e2a47da850 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/1804840938 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5f4050040 con 0x55e2a47da850 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff60cbe1740 10 client.15894 fetch_fsmap learned FSMap version 29 2024-03-19T01:59:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1804840938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e2a485b820 con 0x55e2a47da850 2024-03-19T01:59:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/1804840938 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec09f8b0 con 0x55e2a47da850 2024-03-19T01:59:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/1804840938 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff5f4057480 con 0x55e2a47da850 2024-03-19T01:59:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff60cbe1740 10 client.15894 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T01:59:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff60cbe1740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T01:59:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff60cbe1740 20 client.15894 populate_metadata read hostname 'smithi092' 2024-03-19T01:59:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e2a47e4e40 0x55e2a485ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T01:59:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff60cbe1740 4 client.15894 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T01:59:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1804840938 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2a474b470 con 0x55e2a47e4e40 2024-03-19T01:59:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.238+0000 7ff602ffd640 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e2a47e4e40 0x55e2a485ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T01:59:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.239+0000 7ff5e2ffd640 10 client.15894 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.239+0000 7ff602ffd640 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e2a47e4e40 0x55e2a485ddc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db3271032c6ce6d8 server_cookie=4e3eff10dabcc4c6 in_seq=0 out_seq=0 2024-03-19T01:59:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff5e2ffd640 1 -- 172.21.15.92:0/1804840938 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2a474b470 con 0x55e2a47e4e40 2024-03-19T01:59:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff5e2ffd640 10 client.15894 handle_command_reply: tid=0 2024-03-19T01:59:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 client.15894 shutdown 2024-03-19T01:59:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1804840938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47da850 msgr2=0x55e2a47ee180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47da850 0x55e2a47ee180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff5f402cdd0 tx=0x7ff5f4032a80 comp rx=0 tx=0).stop 2024-03-19T01:59:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1804840938 shutdown_connections 2024-03-19T01:59:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55e2a47e4e40 0x55e2a485ddc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e2a47dfcd0 0x55e2a47ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff5e2ffd640 0 client.15894 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T01:59:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a47df2e0 0x55e2a47ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 --2- 172.21.15.92:0/1804840938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e2a47da850 0x55e2a47ee180 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1804840938 >> 172.21.15.92:0/1804840938 conn(0x55e2a47e8ea0 msgr2=0x55e2a47e0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1804840938 shutdown_connections 2024-03-19T01:59:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 1 -- 172.21.15.92:0/1804840938 wait complete. 2024-03-19T01:59:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.245+0000 7ff60cbe1740 20 client.15894 trim_cache size 0 max 16384 2024-03-19T01:59:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T01:59:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T01:59:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T01:59:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T01:59:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 -- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5f00a27f0 msgr2=0x7ff5f00a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5f00a27f0 0x7ff5f00a4cb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4006010 tx=0x7ff5f4032590 comp rx=0 tx=0).stop 2024-03-19T01:59:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 -- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff604104a20 msgr2=0x7ff604197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T01:59:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff604104a20 0x7ff604197ec0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00e9c0 tx=0x7ff5ec00ee80 comp rx=0 tx=0).stop 2024-03-19T01:59:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 -- 172.21.15.92:0/1677046082 shutdown_connections 2024-03-19T01:59:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604107350 0x7ff60419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff604104a20 0x7ff604197ec0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff5f00a27f0 0x7ff5f00a4cb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 --2- 172.21.15.92:0/1677046082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6041020f0 0x7ff604197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T01:59:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 -- 172.21.15.92:0/1677046082 >> 172.21.15.92:0/1677046082 conn(0x7ff6040fbb00 msgr2=0x7ff604102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T01:59:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 -- 172.21.15.92:0/1677046082 shutdown_connections 2024-03-19T01:59:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T01:59:55.246+0000 7ff608d9d640 1 -- 172.21.15.92:0/1677046082 wait complete. 2024-03-19T01:59:55.269 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:313/600: open fd = 3 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:314/600: open fd = 3 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:315/600: open fd = 3 2024-03-19T01:59:55.270 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:316/600: open fd = 3 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:317/600: open fd = 3 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:318/600: open fd = 3 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.271 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:319/600: open fd = 3 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:320/600: open fd = 3 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:321/600: open fd = 3 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:322/600: open fd = 3 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.272 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:323/600: open fd = 3 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:324/600: open fd = 3 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:325/600: open fd = 3 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:326/600: open fd = 3 2024-03-19T01:59:55.273 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:327/600: open fd = 3 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:328/600: open fd = 3 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:329/600: open fd = 3 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.274 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:330/600: open fd = 3 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:331/600: open fd = 3 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:332/600: open fd = 3 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.275 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:333/600: open fd = 3 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:334/600: open fd = 3 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:335/600: open fd = 3 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:336/600: open fd = 3 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.276 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:337/600: open fd = 3 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:338/600: open fd = 3 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:339/600: open fd = 3 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.277 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:340/600: open fd = 3 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:341/600: open fd = 3 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:342/600: open fd = 3 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.278 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.279 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.279 INFO:tasks.workunit.client.0.smithi092.stdout:343/600: open fd = 3 2024-03-19T01:59:55.279 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.279 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.279 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.279 INFO:tasks.workunit.client.0.smithi092.stdout:344/600: open fd = 3 2024-03-19T01:59:55.280 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.280 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.280 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.280 INFO:tasks.workunit.client.0.smithi092.stdout:345/600: open fd = 3 2024-03-19T01:59:55.280 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.280 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.280 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.280 INFO:tasks.workunit.client.0.smithi092.stdout:346/600: open fd = 3 2024-03-19T01:59:55.280 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:347/600: open fd = 3 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:348/600: open fd = 3 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:349/600: open fd = 3 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.281 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:350/600: open fd = 3 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:351/600: open fd = 3 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:352/600: open fd = 3 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.282 INFO:tasks.workunit.client.0.smithi092.stdout:353/600: open fd = 3 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:354/600: open fd = 3 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:355/600: open fd = 3 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:356/600: open fd = 3 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.283 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:357/600: open fd = 3 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:358/600: open fd = 3 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:359/600: open fd = 3 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:360/600: open fd = 3 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.284 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:361/600: open fd = 3 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:362/600: open fd = 3 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:363/600: open fd = 3 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:364/600: open fd = 3 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.285 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:365/600: open fd = 3 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:366/600: open fd = 3 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:367/600: open fd = 3 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.286 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:368/600: open fd = 3 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:369/600: open fd = 3 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:370/600: open fd = 3 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:371/600: open fd = 3 2024-03-19T01:59:55.287 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:372/600: open fd = 3 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:373/600: open fd = 3 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:374/600: open fd = 3 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T01:59:55.288 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T01:59:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:56 smithi092 ceph-mon[62574]: pgmap v565: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T01:59:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:56 smithi144 ceph-mon[67199]: pgmap v565: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T01:59:57.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:56 smithi159 ceph-mon[67110]: pgmap v565: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T01:59:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:58 smithi092 ceph-mon[62574]: pgmap v566: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T01:59:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:58 smithi144 ceph-mon[67199]: pgmap v566: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T01:59:59.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:58 smithi159 ceph-mon[67110]: pgmap v566: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T02:00:00.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 01:59:59 smithi092 ceph-mon[62574]: pgmap v567: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:00.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 01:59:59 smithi144 ceph-mon[67199]: pgmap v567: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:00.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 01:59:59 smithi159 ceph-mon[67110]: pgmap v567: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:00 smithi092 ceph-mon[62574]: overall HEALTH_OK 2024-03-19T02:00:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:00 smithi144 ceph-mon[67199]: overall HEALTH_OK 2024-03-19T02:00:01.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:00 smithi159 ceph-mon[67110]: overall HEALTH_OK 2024-03-19T02:00:02.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:01 smithi092 ceph-mon[62574]: pgmap v568: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:00:02.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:01 smithi144 ceph-mon[67199]: pgmap v568: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:00:02.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:01 smithi159 ceph-mon[67110]: pgmap v568: 97 pgs: 97 active+clean; 2.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:00:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:04 smithi159 ceph-mon[67110]: pgmap v569: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:05.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:04 smithi092 ceph-mon[62574]: pgmap v569: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:04 smithi144 ceph-mon[67199]: pgmap v569: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:06 smithi159 ceph-mon[67110]: pgmap v570: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-19T02:00:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:06 smithi092 ceph-mon[62574]: pgmap v570: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-19T02:00:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:06 smithi144 ceph-mon[67199]: pgmap v570: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-19T02:00:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.332+0000 7f15a2c73640 1 -- 172.21.15.92:0/4177831986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f159c100740 msgr2=0x7f159c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.332+0000 7f15a2c73640 1 --2- 172.21.15.92:0/4177831986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f159c100740 0x7f159c100b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1590009860 tx=0x7f159002bb70 comp rx=0 tx=0).stop 2024-03-19T02:00:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.332+0000 7f15a2c73640 1 -- 172.21.15.92:0/4177831986 shutdown_connections 2024-03-19T02:00:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.332+0000 7f15a2c73640 1 --2- 172.21.15.92:0/4177831986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c1022f0 0x7f159c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.332+0000 7f15a2c73640 1 --2- 172.21.15.92:0/4177831986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c101930 0x7f159c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.332+0000 7f15a2c73640 1 --2- 172.21.15.92:0/4177831986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f159c100740 0x7f159c100b60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 -- 172.21.15.92:0/4177831986 >> 172.21.15.92:0/4177831986 conn(0x7f159c0fbcb0 msgr2=0x7f159c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:08.335 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 -- 172.21.15.92:0/4177831986 shutdown_connections 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 -- 172.21.15.92:0/4177831986 wait complete. 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 Processor -- start 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 -- start start 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c100740 0x7f159c1047a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c101930 0x7f159c102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f159c1022f0 0x7f159c103320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f159c10c390 con 0x7f159c101930 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f159c10c210 con 0x7f159c100740 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a2c73640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f159c10c510 con 0x7f159c1022f0 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f159bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c101930 0x7f159c102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a0a0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c100740 0x7f159c1047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f159bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c101930 0x7f159c102de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54600/0 (socket says 172.21.15.92:54600) 2024-03-19T02:00:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a0a0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c100740 0x7f159c1047a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:60786/0 (socket says 172.21.15.92:60786) 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f159bfff640 1 -- 172.21.15.92:0/3321270305 learned_addr learned my addr 172.21.15.92:0/3321270305 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.333+0000 7f15a1210640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f159c1022f0 0x7f159c103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f159bfff640 1 -- 172.21.15.92:0/3321270305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f159c1022f0 msgr2=0x7f159c103320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f159bfff640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f159c1022f0 0x7f159c103320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f159bfff640 1 -- 172.21.15.92:0/3321270305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c100740 msgr2=0x7f159c1047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f159bfff640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c100740 0x7f159c1047a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f159bfff640 1 -- 172.21.15.92:0/3321270305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c103bb0 con 0x7f159c101930 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c100740 0x7f159c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f159bfff640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c101930 0x7f159c102de0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f158800e9f0 tx=0x7f158800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f1599ffb640 1 -- 172.21.15.92:0/3321270305 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158800cd80 con 0x7f159c101930 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f15a2c73640 1 -- 172.21.15.92:0/3321270305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159c1a1c00 con 0x7f159c101930 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f1599ffb640 1 -- 172.21.15.92:0/3321270305 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1588010910 con 0x7f159c101930 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f1599ffb640 1 -- 172.21.15.92:0/3321270305 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588018950 con 0x7f159c101930 2024-03-19T02:00:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.334+0000 7f15a2c73640 1 -- 172.21.15.92:0/3321270305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159c1a21a0 con 0x7f159c101930 2024-03-19T02:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a585e740 1 Processor -- start 2024-03-19T02:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a585e740 1 -- start start 2024-03-19T02:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a585e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159e4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a585e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 0x55ff159e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a585e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 0x55ff159ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a585e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ff15948be0 con 0x55ff159e4100 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a585e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff158f1960 con 0x55ff159e4af0 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a585e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff15944c80 con 0x55ff159e9c60 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a1210640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a1210640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159e4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54612/0 (socket says 172.21.15.92:54612) 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a1210640 1 -- 172.21.15.92:0/2029945804 learned_addr learned my addr 172.21.15.92:0/2029945804 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f1599ffb640 1 -- 172.21.15.92:0/3321270305 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f1588010ac0 con 0x7f159c101930 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 0x55ff159e95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f159bfff640 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 0x55ff159ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a1210640 1 -- 172.21.15.92:0/2029945804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 msgr2=0x55ff159ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a1210640 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 0x55ff159ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a1210640 1 -- 172.21.15.92:0/2029945804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 msgr2=0x55ff159e95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a1210640 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 0x55ff159e95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a1210640 1 -- 172.21.15.92:0/2029945804 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff158e8d20 con 0x55ff159e4100 2024-03-19T02:00:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f159bfff640 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 0x55ff159ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.337+0000 7f15a1210640 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159e4520 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f159400e450 tx=0x7f15940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217a5cea4aeb3f47 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.338+0000 7f1599ffb640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f158c0a28a0 0x7f158c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.338+0000 7f1586ffd640 1 -- 172.21.15.92:0/2029945804 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159400d910 con 0x55ff159e4100 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.338+0000 7f1586ffd640 1 -- 172.21.15.92:0/2029945804 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159401c070 con 0x55ff159e4100 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.338+0000 7f1586ffd640 1 -- 172.21.15.92:0/2029945804 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594016380 con 0x55ff159e4100 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.338+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f158c0a28a0 0x7f158c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.338+0000 7f1599ffb640 1 -- 172.21.15.92:0/3321270305 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f15880cee30 con 0x7f159c101930 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.338+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f158c0a28a0 0x7f158c0a4d60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1590009830 tx=0x7f159002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.338+0000 7f15a585e740 1 -- 172.21.15.92:0/2029945804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 msgr2=0x55ff159e4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.338+0000 7f15a585e740 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159e4520 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f159400e450 tx=0x7f15940095c0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 -- 172.21.15.92:0/2029945804 shutdown_connections 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 0x55ff159ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 0x55ff159e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 --2- 172.21.15.92:0/2029945804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159e4520 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 -- 172.21.15.92:0/2029945804 >> 172.21.15.92:0/2029945804 conn(0x55ff159df670 msgr2=0x55ff159dfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 -- 172.21.15.92:0/2029945804 shutdown_connections 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 -- 172.21.15.92:0/2029945804 wait complete. 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 -- start start 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 Processor -- start 2024-03-19T02:00:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4af0 0x55ff159ee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e9c60 0x55ff159f5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ff159ecc10 con 0x55ff159e4100 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff15904db0 con 0x55ff159e9c60 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.339+0000 7f15a585e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff159ecd90 con 0x55ff159e4af0 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a585e740 10 client.0 _ll_register_callbacks cb 0x55ff15909e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a0a0f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f159bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4af0 0x55ff159ee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a0a0f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159ee0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54622/0 (socket says 172.21.15.92:54622) 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f159bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4af0 0x55ff159ee5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:34594/0 (socket says 172.21.15.92:34594) 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a1210640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e9c60 0x55ff159f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a0a0f640 1 -- 172.21.15.92:0/3251871024 learned_addr learned my addr 172.21.15.92:0/3251871024 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a0a0f640 1 -- 172.21.15.92:0/3251871024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4af0 msgr2=0x55ff159ee5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4af0 0x55ff159ee5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a0a0f640 1 -- 172.21.15.92:0/3251871024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e9c60 msgr2=0x55ff159f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e9c60 0x55ff159f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a0a0f640 1 -- 172.21.15.92:0/3251871024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff15903c60 con 0x55ff159e4100 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f159bfff640 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4af0 0x55ff159ee5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a1210640 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e9c60 0x55ff159f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159ee0b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f159002c180 tx=0x7f159002bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3da3bdea9f004c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f157bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f157bfff640 1 -- 172.21.15.92:0/3251871024 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590039040 con 0x55ff159e4100 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f157bfff640 1 -- 172.21.15.92:0/3251871024 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f159002c510 con 0x55ff159e4100 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a585e740 1 -- 172.21.15.92:0/3251871024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159401c070 con 0x55ff159e4100 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f157bfff640 1 -- 172.21.15.92:0/3251871024 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f159002c710 con 0x55ff159e4100 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a585e740 10 client.15906 fetch_fsmap learned FSMap version 29 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.340+0000 7f15a585e740 1 -- 172.21.15.92:0/3251871024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff1565d210 con 0x55ff159e4100 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f157bfff640 1 -- 172.21.15.92:0/3251871024 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159401c070 con 0x55ff159e4100 2024-03-19T02:00:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f157bfff640 1 -- 172.21.15.92:0/3251871024 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1590057070 con 0x55ff159e4100 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f15a585e740 10 client.15906 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f15a585e740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f15a585e740 20 client.15906 populate_metadata read hostname 'smithi092' 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f15a585e740 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ff159f9e80 0x55ff159fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f15a585e740 4 client.15906 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f15a585e740 1 -- 172.21.15.92:0/3251871024 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff15956120 con 0x55ff159f9e80 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f159bfff640 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ff159f9e80 0x55ff159fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f157bfff640 10 client.15906 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.341+0000 7f159bfff640 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ff159f9e80 0x55ff159fc2b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f286c7d7133c940e server_cookie=987b821ec74be1ea in_seq=0 out_seq=0 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.342+0000 7f157bfff640 1 -- 172.21.15.92:0/3251871024 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ff15956120 con 0x55ff159f9e80 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.342+0000 7f157bfff640 10 client.15906 handle_command_reply: tid=0 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.342+0000 7f15a585e740 1 client.15906 shutdown 2024-03-19T02:00:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.342+0000 7f15a585e740 1 -- 172.21.15.92:0/3251871024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 msgr2=0x55ff159ee0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.342+0000 7f15a585e740 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159ee0b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f159002c180 tx=0x7f159002bb70 comp rx=0 tx=0).stop 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f15a585e740 1 -- 172.21.15.92:0/3251871024 shutdown_connections 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f15a585e740 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ff159f9e80 0x55ff159fc2b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f15a585e740 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e9c60 0x55ff159f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f15a585e740 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4af0 0x55ff159ee5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f157bfff640 0 client.15906 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f15a585e740 1 --2- 172.21.15.92:0/3251871024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4100 0x55ff159ee0b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f15a585e740 1 -- 172.21.15.92:0/3251871024 >> 172.21.15.92:0/3251871024 conn(0x55ff159df670 msgr2=0x55ff159e08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f15a585e740 1 -- 172.21.15.92:0/3251871024 shutdown_connections 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f15a585e740 1 -- 172.21.15.92:0/3251871024 wait complete. 2024-03-19T02:00:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.343+0000 7f15a585e740 20 client.15906 trim_cache size 0 max 16384 2024-03-19T02:00:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.357+0000 7f15a585e740 1 Processor -- start 2024-03-19T02:00:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.357+0000 7f15a585e740 1 -- start start 2024-03-19T02:00:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a585e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4100 0x55ff159deab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a585e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 0x55ff159f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a585e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e9c60 0x55ff159e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a585e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1aaef0 con 0x55ff159e9c60 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a585e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1aca30 con 0x55ff159e4af0 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a585e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1ab5c0 con 0x55ff159e4100 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a1210640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e9c60 0x55ff159e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a1210640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e9c60 0x55ff159e2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54624/0 (socket says 172.21.15.92:54624) 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a0a0f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4100 0x55ff159deab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a1210640 1 -- 172.21.15.92:0/3890440143 learned_addr learned my addr 172.21.15.92:0/3890440143 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f159bfff640 1 --2- 172.21.15.92:0/3890440143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 0x55ff159f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a1210640 1 -- 172.21.15.92:0/3890440143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4100 msgr2=0x55ff159deab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a1210640 1 --2- 172.21.15.92:0/3890440143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4100 0x55ff159deab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a1210640 1 -- 172.21.15.92:0/3890440143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 msgr2=0x55ff159f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a1210640 1 --2- 172.21.15.92:0/3890440143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 0x55ff159f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a1210640 1 -- 172.21.15.92:0/3890440143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff15729620 con 0x55ff159e9c60 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15a1210640 1 --2- 172.21.15.92:0/3890440143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e9c60 0x55ff159e2fe0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f159400e450 tx=0x7f15940099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdb7406b9d13861 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15867fc640 1 -- 172.21.15.92:0/3890440143 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594003b70 con 0x55ff159e9c60 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15867fc640 1 -- 172.21.15.92:0/3890440143 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159401c030 con 0x55ff159e9c60 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.358+0000 7f15867fc640 1 -- 172.21.15.92:0/3890440143 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159400dce0 con 0x55ff159e9c60 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 -- 172.21.15.92:0/3890440143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e9c60 msgr2=0x55ff159e2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 --2- 172.21.15.92:0/3890440143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e9c60 0x55ff159e2fe0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f159400e450 tx=0x7f15940099c0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 -- 172.21.15.92:0/3890440143 shutdown_connections 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 --2- 172.21.15.92:0/3890440143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e9c60 0x55ff159e2fe0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 --2- 172.21.15.92:0/3890440143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4af0 0x55ff159f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 --2- 172.21.15.92:0/3890440143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e4100 0x55ff159deab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 -- 172.21.15.92:0/3890440143 >> 172.21.15.92:0/3890440143 conn(0x55ff159df670 msgr2=0x55ff159e28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 -- 172.21.15.92:0/3890440143 shutdown_connections 2024-03-19T02:00:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 -- 172.21.15.92:0/3890440143 wait complete. 2024-03-19T02:00:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.359+0000 7f15a585e740 1 -- start start 2024-03-19T02:00:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a585e740 1 Processor -- start 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a585e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4100 0x55ff159f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a585e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4af0 0x55ff159f3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a585e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 0x55ff159f4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a585e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1ac360 con 0x55ff159e4af0 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a585e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1abc90 con 0x55ff159e4100 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a585e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1a8ed0 con 0x55ff159e9c60 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a585e740 10 client.0 _ll_register_callbacks cb 0x7f1588005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f159bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4af0 0x55ff159f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f159bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4af0 0x55ff159f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54632/0 (socket says 172.21.15.92:54632) 2024-03-19T02:00:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f159bfff640 1 -- 172.21.15.92:0/3526857478 learned_addr learned my addr 172.21.15.92:0/3526857478 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4100 0x55ff159f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a1210640 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 0x55ff159f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f159bfff640 1 -- 172.21.15.92:0/3526857478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 msgr2=0x55ff159f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f159bfff640 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 0x55ff159f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f159bfff640 1 -- 172.21.15.92:0/3526857478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4100 msgr2=0x55ff159f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f159bfff640 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4100 0x55ff159f2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f159bfff640 1 -- 172.21.15.92:0/3526857478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff159f4730 con 0x55ff159e4af0 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.360+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4100 0x55ff159f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f159bfff640 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4af0 0x55ff159f3130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f158809b590 tx=0x7f1588086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6049de5da24cc52 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f157bfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f157bfff640 1 -- 172.21.15.92:0/3526857478 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15880b3020 con 0x55ff159e4af0 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f157bfff640 1 -- 172.21.15.92:0/3526857478 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f15880cce00 con 0x55ff159e4af0 2024-03-19T02:00:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a585e740 1 -- 172.21.15.92:0/3526857478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159401c030 con 0x55ff159e4af0 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f157bfff640 1 -- 172.21.15.92:0/3526857478 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1588099b90 con 0x55ff159e4af0 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a585e740 10 client.15912 fetch_fsmap learned FSMap version 29 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a585e740 1 -- 172.21.15.92:0/3526857478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff15a601f0 con 0x55ff159e4af0 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f157bfff640 1 -- 172.21.15.92:0/3526857478 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159401c030 con 0x55ff159e4af0 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f157bfff640 1 -- 172.21.15.92:0/3526857478 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f15880afbc0 con 0x55ff159e4af0 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a585e740 10 client.15912 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a585e740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a585e740 20 client.15912 populate_metadata read hostname 'smithi092' 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a585e740 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ff15a60ac0 0x55ff15a62ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a585e740 4 client.15912 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a585e740 1 -- 172.21.15.92:0/3526857478 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff15950470 con 0x55ff15a60ac0 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.361+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ff15a60ac0 0x55ff15a62ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.362+0000 7f157bfff640 10 client.15912 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.362+0000 7f15a0a0f640 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ff15a60ac0 0x55ff15a62ee0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6dcd1c03b2da071 server_cookie=26016d54b75c8235 in_seq=0 out_seq=0 2024-03-19T02:00:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f157bfff640 1 -- 172.21.15.92:0/3526857478 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff15950470 con 0x55ff15a60ac0 2024-03-19T02:00:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f157bfff640 10 client.15912 handle_command_reply: tid=0 2024-03-19T02:00:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 client.15912 shutdown 2024-03-19T02:00:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 -- 172.21.15.92:0/3526857478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4af0 msgr2=0x55ff159f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4af0 0x55ff159f3130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f158809b590 tx=0x7f1588086040 comp rx=0 tx=0).stop 2024-03-19T02:00:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 -- 172.21.15.92:0/3526857478 shutdown_connections 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55ff15a60ac0 0x55ff15a62ee0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff159e9c60 0x55ff159f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff159e4af0 0x55ff159f3130 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f157bfff640 0 client.15912 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 --2- 172.21.15.92:0/3526857478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff159e4100 0x55ff159f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 -- 172.21.15.92:0/3526857478 >> 172.21.15.92:0/3526857478 conn(0x55ff159df670 msgr2=0x55ff159e3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 -- 172.21.15.92:0/3526857478 shutdown_connections 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 1 -- 172.21.15.92:0/3526857478 wait complete. 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.368+0000 7f15a585e740 20 client.15912 trim_cache size 0 max 16384 2024-03-19T02:00:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:00:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:00:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:00:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 -- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f158c0a28a0 msgr2=0x7f158c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f158c0a28a0 0x7f158c0a4d60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1590009830 tx=0x7f159002e040 comp rx=0 tx=0).stop 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 -- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c101930 msgr2=0x7f159c102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c101930 0x7f159c102de0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f158800e9f0 tx=0x7f158800eeb0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 -- 172.21.15.92:0/3321270305 shutdown_connections 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f158c0a28a0 0x7f158c0a4d60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f159c1022f0 0x7f159c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c101930 0x7f159c102de0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 --2- 172.21.15.92:0/3321270305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c100740 0x7f159c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 -- 172.21.15.92:0/3321270305 >> 172.21.15.92:0/3321270305 conn(0x7f159c0fbcb0 msgr2=0x7f159c068ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 -- 172.21.15.92:0/3321270305 shutdown_connections 2024-03-19T02:00:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:08.369+0000 7f15a2c73640 1 -- 172.21.15.92:0/3321270305 wait complete. 2024-03-19T02:00:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:08 smithi159 ceph-mon[67110]: pgmap v571: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-03-19T02:00:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:08 smithi092 ceph-mon[62574]: pgmap v571: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-03-19T02:00:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:08 smithi144 ceph-mon[67199]: pgmap v571: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-03-19T02:00:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:10 smithi159 ceph-mon[67110]: pgmap v572: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 18 op/s 2024-03-19T02:00:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:10 smithi092 ceph-mon[62574]: pgmap v572: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 18 op/s 2024-03-19T02:00:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:10 smithi144 ceph-mon[67199]: pgmap v572: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 18 op/s 2024-03-19T02:00:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:12 smithi159 ceph-mon[67110]: pgmap v573: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-03-19T02:00:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:12 smithi092 ceph-mon[62574]: pgmap v573: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-03-19T02:00:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:12 smithi144 ceph-mon[67199]: pgmap v573: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-03-19T02:00:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:14 smithi159 ceph-mon[67110]: pgmap v574: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 20 op/s 2024-03-19T02:00:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:14 smithi092 ceph-mon[62574]: pgmap v574: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 20 op/s 2024-03-19T02:00:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:14 smithi144 ceph-mon[67199]: pgmap v574: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 20 op/s 2024-03-19T02:00:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:16 smithi159 ceph-mon[67110]: pgmap v575: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:00:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:16 smithi092 ceph-mon[62574]: pgmap v575: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:00:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:16 smithi144 ceph-mon[67199]: pgmap v575: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:00:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: pgmap v576: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:00:18.956 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:18 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:00:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: pgmap v576: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:00:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:00:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:00:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:00:19.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:18 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:00:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: pgmap v576: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:00:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:00:19.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:18 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:00:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:19 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-19T02:00:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:19 smithi159 ceph-mon[67110]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-19T02:00:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:19 smithi159 ceph-mon[67110]: Adjusting osd_memory_target on smithi159 to 174.1M 2024-03-19T02:00:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:19 smithi159 ceph-mon[67110]: Unable to set osd_memory_target on smithi159 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-19T02:00:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:19 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-19T02:00:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:19 smithi092 ceph-mon[62574]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-19T02:00:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:19 smithi092 ceph-mon[62574]: Adjusting osd_memory_target on smithi159 to 174.1M 2024-03-19T02:00:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:19 smithi092 ceph-mon[62574]: Unable to set osd_memory_target on smithi159 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-19T02:00:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:19 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi144 to 174.1M 2024-03-19T02:00:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:19 smithi144 ceph-mon[67199]: Unable to set osd_memory_target on smithi144 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-03-19T02:00:20.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:19 smithi144 ceph-mon[67199]: Adjusting osd_memory_target on smithi159 to 174.1M 2024-03-19T02:00:20.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:19 smithi144 ceph-mon[67199]: Unable to set osd_memory_target on smithi159 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-03-19T02:00:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:20 smithi159 ceph-mon[67110]: pgmap v577: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:00:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:20 smithi092 ceph-mon[62574]: pgmap v577: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:00:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:20 smithi144 ceph-mon[67199]: pgmap v577: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:00:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 -- 172.21.15.92:0/828659620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c100740 msgr2=0x7ffa4c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 --2- 172.21.15.92:0/828659620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c100740 0x7ffa4c100b60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009900 tx=0x7ffa4402bc30 comp rx=0 tx=0).stop 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 -- 172.21.15.92:0/828659620 shutdown_connections 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 --2- 172.21.15.92:0/828659620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1022f0 0x7ffa4c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 --2- 172.21.15.92:0/828659620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa4c101930 0x7ffa4c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 --2- 172.21.15.92:0/828659620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c100740 0x7ffa4c100b60 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 -- 172.21.15.92:0/828659620 >> 172.21.15.92:0/828659620 conn(0x7ffa4c0fbcb0 msgr2=0x7ffa4c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 -- 172.21.15.92:0/828659620 shutdown_connections 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 -- 172.21.15.92:0/828659620 wait complete. 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.457+0000 7ffa545bc640 1 Processor -- start 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa545bc640 1 -- start start 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa545bc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa4c100740 0x7ffa4c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa545bc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c101930 0x7ffa4c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa545bc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1022f0 0x7ffa4c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa545bc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c10c390 con 0x7ffa4c101930 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa545bc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c10c210 con 0x7ffa4c1022f0 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa545bc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c10c510 con 0x7ffa4c100740 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa51b57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c101930 0x7ffa4c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa51b57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c101930 0x7ffa4c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59396/0 (socket says 172.21.15.92:59396) 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa51b57640 1 -- 172.21.15.92:0/1297210215 learned_addr learned my addr 172.21.15.92:0/1297210215 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa52358640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa4c100740 0x7ffa4c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa52b59640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1022f0 0x7ffa4c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa51b57640 1 -- 172.21.15.92:0/1297210215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa4c100740 msgr2=0x7ffa4c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa51b57640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa4c100740 0x7ffa4c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa51b57640 1 -- 172.21.15.92:0/1297210215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1022f0 msgr2=0x7ffa4c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa51b57640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1022f0 0x7ffa4c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa51b57640 1 -- 172.21.15.92:0/1297210215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c19f720 con 0x7ffa4c101930 2024-03-19T02:00:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa51b57640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c101930 0x7ffa4c198040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9f0 tx=0x7ffa3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa437fe640 1 -- 172.21.15.92:0/1297210215 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00cd80 con 0x7ffa4c101930 2024-03-19T02:00:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa437fe640 1 -- 172.21.15.92:0/1297210215 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa3c010910 con 0x7ffa4c101930 2024-03-19T02:00:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa545bc640 1 -- 172.21.15.92:0/1297210215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c19fa10 con 0x7ffa4c101930 2024-03-19T02:00:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa437fe640 1 -- 172.21.15.92:0/1297210215 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c018990 con 0x7ffa4c101930 2024-03-19T02:00:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.458+0000 7ffa545bc640 1 -- 172.21.15.92:0/1297210215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c19ff60 con 0x7ffa4c101930 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.461+0000 7ffa437fe640 1 -- 172.21.15.92:0/1297210215 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ffa3c010ac0 con 0x7ffa4c101930 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.461+0000 7ffa5719b740 1 Processor -- start 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.461+0000 7ffa5719b740 1 -- start start 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.461+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5e100 0x55d632e5e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.461+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 0x55d632e635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.461+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e63c60 0x55d632e660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa5719b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d632dc2be0 con 0x55d632e63c60 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa5719b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d632d6b960 con 0x55d632e5eaf0 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa5719b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d632dbec80 con 0x55d632e5e100 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa437fe640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffa380a28a0 0x7ffa380a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5e100 0x55d632e5e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa51b57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e63c60 0x55d632e660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5e100 0x55d632e5e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:58006/0 (socket says 172.21.15.92:58006) 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 -- 172.21.15.92:0/2576171959 learned_addr learned my addr 172.21.15.92:0/2576171959 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52358640 1 --2- 172.21.15.92:0/2576171959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 0x55d632e635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffa380a28a0 0x7ffa380a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa437fe640 1 -- 172.21.15.92:0/1297210215 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ffa3c0ced80 con 0x7ffa4c101930 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 -- 172.21.15.92:0/2576171959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 msgr2=0x55d632e635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 --2- 172.21.15.92:0/2576171959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 0x55d632e635f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 -- 172.21.15.92:0/2576171959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e63c60 msgr2=0x55d632e660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 --2- 172.21.15.92:0/2576171959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e63c60 0x55d632e660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 -- 172.21.15.92:0/2576171959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d632d62d20 con 0x55d632e5e100 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 --2- 172.21.15.92:0/2576171959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5e100 0x55d632e5e520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa48005c40 tx=0x7ffa4800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fcb886637a2f7ee server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa52b59640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffa380a28a0 0x7ffa380a4d60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa48008000 tx=0x7ffa4801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.462+0000 7ffa40bfe640 1 -- 172.21.15.92:0/2576171959 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800fd30 con 0x55d632e5e100 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa40bfe640 1 -- 172.21.15.92:0/2576171959 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa48022030 con 0x55d632e5e100 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa40bfe640 1 -- 172.21.15.92:0/2576171959 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48010930 con 0x55d632e5e100 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa5719b740 1 -- 172.21.15.92:0/2576171959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5e100 msgr2=0x55d632e5e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa5719b740 1 --2- 172.21.15.92:0/2576171959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5e100 0x55d632e5e520 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa48005c40 tx=0x7ffa4800f3c0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa5719b740 1 -- 172.21.15.92:0/2576171959 shutdown_connections 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa5719b740 1 --2- 172.21.15.92:0/2576171959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e63c60 0x55d632e660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa5719b740 1 --2- 172.21.15.92:0/2576171959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 0x55d632e635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa5719b740 1 --2- 172.21.15.92:0/2576171959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5e100 0x55d632e5e520 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa5719b740 1 -- 172.21.15.92:0/2576171959 >> 172.21.15.92:0/2576171959 conn(0x55d632e59670 msgr2=0x55d632e59a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa5719b740 1 -- 172.21.15.92:0/2576171959 shutdown_connections 2024-03-19T02:00:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.463+0000 7ffa5719b740 1 -- 172.21.15.92:0/2576171959 wait complete. 2024-03-19T02:00:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa5719b740 1 -- start start 2024-03-19T02:00:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa5719b740 1 Processor -- start 2024-03-19T02:00:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e68090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 0x55d632e685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 0x55d632e6f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa5719b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d632e66bb0 con 0x55d632e5e100 2024-03-19T02:00:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa5719b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d632d7edb0 con 0x55d632e63c60 2024-03-19T02:00:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa5719b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d632e66d30 con 0x55d632e5eaf0 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa5719b740 10 client.0 _ll_register_callbacks cb 0x55d632d83e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa52358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e68090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa52358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e68090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59420/0 (socket says 172.21.15.92:59420) 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa52358640 1 -- 172.21.15.92:0/3861919468 learned_addr learned my addr 172.21.15.92:0/3861919468 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa51b57640 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 0x55d632e685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa52b59640 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 0x55d632e6f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa52358640 1 -- 172.21.15.92:0/3861919468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 msgr2=0x55d632e685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa52358640 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 0x55d632e685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa52358640 1 -- 172.21.15.92:0/3861919468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 msgr2=0x55d632e6f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa52358640 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 0x55d632e6f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.464+0000 7ffa52358640 1 -- 172.21.15.92:0/3861919468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d632d7dc60 con 0x55d632e5e100 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa52358640 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e68090 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009900 tx=0x7ffa44004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e4ecf741719fdb server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:00:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa35ffb640 1 -- 172.21.15.92:0/3861919468 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44005d50 con 0x55d632e5e100 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa5719b740 1 -- 172.21.15.92:0/3861919468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa48022030 con 0x55d632e5e100 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa35ffb640 1 -- 172.21.15.92:0/3861919468 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ffa44005230 con 0x55d632e5e100 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa35ffb640 1 -- 172.21.15.92:0/3861919468 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa44005430 con 0x55d632e5e100 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa5719b740 10 client.15924 fetch_fsmap learned FSMap version 29 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa5719b740 1 -- 172.21.15.92:0/3861919468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d632ad7210 con 0x55d632e5e100 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa35ffb640 1 -- 172.21.15.92:0/3861919468 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa48022030 con 0x55d632e5e100 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa35ffb640 1 -- 172.21.15.92:0/3861919468 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffa440037b0 con 0x55d632e5e100 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa5719b740 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa5719b740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa5719b740 20 client.15924 populate_metadata read hostname 'smithi092' 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d632e73ef0 0x55d632e76320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa5719b740 4 client.15924 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:00:21.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.465+0000 7ffa5719b740 1 -- 172.21.15.92:0/3861919468 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d632dd0120 con 0x55d632e73ef0 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.466+0000 7ffa51b57640 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d632e73ef0 0x55d632e76320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.466+0000 7ffa51b57640 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d632e73ef0 0x55d632e76320 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1f541ca719995f server_cookie=81813ad21076c716 in_seq=0 out_seq=0 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.466+0000 7ffa35ffb640 10 client.15924 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa35ffb640 1 -- 172.21.15.92:0/3861919468 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d632dd0120 con 0x55d632e73ef0 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa35ffb640 10 client.15924 handle_command_reply: tid=0 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 client.15924 shutdown 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 -- 172.21.15.92:0/3861919468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 msgr2=0x55d632e68090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e68090 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009900 tx=0x7ffa44004250 comp rx=0 tx=0).stop 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 -- 172.21.15.92:0/3861919468 shutdown_connections 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d632e73ef0 0x55d632e76320 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 0x55d632e6f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa35ffb640 0 client.15924 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 0x55d632e685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3861919468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e68090 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 -- 172.21.15.92:0/3861919468 >> 172.21.15.92:0/3861919468 conn(0x55d632e59670 msgr2=0x55d632e5a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 -- 172.21.15.92:0/3861919468 shutdown_connections 2024-03-19T02:00:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 1 -- 172.21.15.92:0/3861919468 wait complete. 2024-03-19T02:00:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.467+0000 7ffa5719b740 20 client.15924 trim_cache size 0 max 16384 2024-03-19T02:00:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.481+0000 7ffa5719b740 1 Processor -- start 2024-03-19T02:00:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.481+0000 7ffa5719b740 1 -- start start 2024-03-19T02:00:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 0x55d632e6f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e63c60 0x55d632e678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa5719b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1a8c20 con 0x55d632e5e100 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa5719b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1aa760 con 0x55d632e5eaf0 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa5719b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1a92f0 con 0x55d632e63c60 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59436/0 (socket says 172.21.15.92:59436) 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52358640 1 -- 172.21.15.92:0/3382998594 learned_addr learned my addr 172.21.15.92:0/3382998594 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52b59640 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e63c60 0x55d632e678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa51b57640 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 0x55d632e6f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52358640 1 -- 172.21.15.92:0/3382998594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e63c60 msgr2=0x55d632e678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52358640 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e63c60 0x55d632e678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52358640 1 -- 172.21.15.92:0/3382998594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 msgr2=0x55d632e6f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52358640 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 0x55d632e6f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52358640 1 -- 172.21.15.92:0/3382998594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d632ba3620 con 0x55d632e5e100 2024-03-19T02:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa51b57640 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 0x55d632e6f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa52358640 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e673a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009850 tx=0x7ffa440045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bc0f0af01056ab server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa37fff640 1 -- 172.21.15.92:0/3382998594 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44005d50 con 0x55d632e5e100 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa37fff640 1 -- 172.21.15.92:0/3382998594 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa440055d0 con 0x55d632e5e100 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.482+0000 7ffa37fff640 1 -- 172.21.15.92:0/3382998594 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44034cb0 con 0x55d632e5e100 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 -- 172.21.15.92:0/3382998594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 msgr2=0x55d632e673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e673a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009850 tx=0x7ffa440045f0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 -- 172.21.15.92:0/3382998594 shutdown_connections 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e63c60 0x55d632e678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e5eaf0 0x55d632e6f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 --2- 172.21.15.92:0/3382998594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e673a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 -- 172.21.15.92:0/3382998594 >> 172.21.15.92:0/3382998594 conn(0x55d632e59670 msgr2=0x55d632e5d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 -- 172.21.15.92:0/3382998594 shutdown_connections 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 -- 172.21.15.92:0/3382998594 wait complete. 2024-03-19T02:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 -- start start 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 Processor -- start 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e6c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 0x55d632e6cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 0x55d632e6ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1aa090 con 0x55d632e5e100 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1a99c0 con 0x55d632e63c60 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1a6c00 con 0x55d632e5eaf0 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa5719b740 10 client.0 _ll_register_callbacks cb 0x7ffa44009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa52358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e6c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa52358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e6c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59450/0 (socket says 172.21.15.92:59450) 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.483+0000 7ffa52358640 1 -- 172.21.15.92:0/200797795 learned_addr learned my addr 172.21.15.92:0/200797795 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa52b59640 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 0x55d632e6ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa51b57640 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 0x55d632e6cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa52358640 1 -- 172.21.15.92:0/200797795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 msgr2=0x55d632e6cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa52358640 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 0x55d632e6cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa52358640 1 -- 172.21.15.92:0/200797795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 msgr2=0x55d632e6ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa52358640 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 0x55d632e6ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa52358640 1 -- 172.21.15.92:0/200797795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d632e6e4c0 con 0x55d632e5e100 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa52358640 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e6c980 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffa4402cda0 tx=0x7ffa44004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9792fba8093af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa35ffb640 1 -- 172.21.15.92:0/200797795 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4405d070 con 0x55d632e5e100 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa35ffb640 1 -- 172.21.15.92:0/200797795 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ffa44003940 con 0x55d632e5e100 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa5719b740 1 -- 172.21.15.92:0/200797795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa440055d0 con 0x55d632e5e100 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa35ffb640 1 -- 172.21.15.92:0/200797795 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa44069050 con 0x55d632e5e100 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa5719b740 10 client.15930 fetch_fsmap learned FSMap version 29 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa5719b740 1 -- 172.21.15.92:0/200797795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d632e6f7d0 con 0x55d632e5e100 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.484+0000 7ffa35ffb640 1 -- 172.21.15.92:0/200797795 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa440055d0 con 0x55d632e5e100 2024-03-19T02:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa35ffb640 1 -- 172.21.15.92:0/200797795 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffa44046c00 con 0x55d632e5e100 2024-03-19T02:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa5719b740 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa5719b740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa5719b740 20 client.15930 populate_metadata read hostname 'smithi092' 2024-03-19T02:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa5719b740 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d632eda7c0 0x55d632edcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa5719b740 4 client.15930 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa5719b740 1 -- 172.21.15.92:0/200797795 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d632dca470 con 0x55d632eda7c0 2024-03-19T02:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa51b57640 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d632eda7c0 0x55d632edcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa35ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.485+0000 7ffa51b57640 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d632eda7c0 0x55d632edcbe0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb1d3670094fd97f server_cookie=1a8b92bc986f7e84 in_seq=0 out_seq=0 2024-03-19T02:00:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.491+0000 7ffa35ffb640 1 -- 172.21.15.92:0/200797795 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d632dca470 con 0x55d632eda7c0 2024-03-19T02:00:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.491+0000 7ffa35ffb640 10 client.15930 handle_command_reply: tid=0 2024-03-19T02:00:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.491+0000 7ffa5719b740 1 client.15930 shutdown 2024-03-19T02:00:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.491+0000 7ffa5719b740 1 -- 172.21.15.92:0/200797795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 msgr2=0x55d632e6c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e6c980 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffa4402cda0 tx=0x7ffa44004250 comp rx=0 tx=0).stop 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 1 -- 172.21.15.92:0/200797795 shutdown_connections 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55d632eda7c0 0x55d632edcbe0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d632e63c60 0x55d632e6ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d632e5eaf0 0x55d632e6cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 1 --2- 172.21.15.92:0/200797795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d632e5e100 0x55d632e6c980 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa35ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 1 -- 172.21.15.92:0/200797795 >> 172.21.15.92:0/200797795 conn(0x55d632e59670 msgr2=0x55d632e5f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 1 -- 172.21.15.92:0/200797795 shutdown_connections 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 1 -- 172.21.15.92:0/200797795 wait complete. 2024-03-19T02:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa5719b740 20 client.15930 trim_cache size 0 max 16384 2024-03-19T02:00:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:00:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:00:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:00:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 -- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffa380a28a0 msgr2=0x7ffa380a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffa380a28a0 0x7ffa380a4d60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa48008000 tx=0x7ffa4801f000 comp rx=0 tx=0).stop 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 -- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c101930 msgr2=0x7ffa4c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c101930 0x7ffa4c198040 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9f0 tx=0x7ffa3c00eeb0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 -- 172.21.15.92:0/1297210215 shutdown_connections 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1022f0 0x7ffa4c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa4c101930 0x7ffa4c198040 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ffa380a28a0 0x7ffa380a4d60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 --2- 172.21.15.92:0/1297210215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa4c100740 0x7ffa4c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 -- 172.21.15.92:0/1297210215 >> 172.21.15.92:0/1297210215 conn(0x7ffa4c0fbcb0 msgr2=0x7ffa4c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 -- 172.21.15.92:0/1297210215 shutdown_connections 2024-03-19T02:00:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:21.492+0000 7ffa545bc640 1 -- 172.21.15.92:0/1297210215 wait complete. 2024-03-19T02:00:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:22 smithi159 ceph-mon[67110]: pgmap v578: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-03-19T02:00:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:22 smithi092 ceph-mon[62574]: pgmap v578: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-03-19T02:00:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:22 smithi144 ceph-mon[67199]: pgmap v578: 97 pgs: 97 active+clean; 2.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-03-19T02:00:24.911 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:24 smithi144 ceph-mon[67199]: pgmap v579: 97 pgs: 97 active+clean; 2.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T02:00:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:24 smithi159 ceph-mon[67110]: pgmap v579: 97 pgs: 97 active+clean; 2.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T02:00:25.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:24 smithi092 ceph-mon[62574]: pgmap v579: 97 pgs: 97 active+clean; 2.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T02:00:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:26 smithi159 ceph-mon[67110]: pgmap v580: 97 pgs: 97 active+clean; 2.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:00:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:26 smithi092 ceph-mon[62574]: pgmap v580: 97 pgs: 97 active+clean; 2.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:00:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:26 smithi144 ceph-mon[67199]: pgmap v580: 97 pgs: 97 active+clean; 2.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:00:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:28 smithi159 ceph-mon[67110]: pgmap v581: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:00:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:28 smithi092 ceph-mon[62574]: pgmap v581: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:00:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:28 smithi144 ceph-mon[67199]: pgmap v581: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:00:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:30 smithi159 ceph-mon[67110]: pgmap v582: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:31.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:30 smithi092 ceph-mon[62574]: pgmap v582: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:30 smithi144 ceph-mon[67199]: pgmap v582: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:32 smithi159 ceph-mon[67110]: pgmap v583: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:00:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:32 smithi092 ceph-mon[62574]: pgmap v583: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:00:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:32 smithi144 ceph-mon[67199]: pgmap v583: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.584+0000 7f01218b8640 1 -- 172.21.15.92:0/3962024205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c1022f0 msgr2=0x7f011c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.584+0000 7f01218b8640 1 --2- 172.21.15.92:0/3962024205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c1022f0 0x7f011c108b80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0114009ca0 tx=0x7f011402c0f0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.584+0000 7f01218b8640 1 -- 172.21.15.92:0/3962024205 shutdown_connections 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.584+0000 7f01218b8640 1 --2- 172.21.15.92:0/3962024205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c1022f0 0x7f011c108b80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.584+0000 7f01218b8640 1 --2- 172.21.15.92:0/3962024205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011c101930 0x7f011c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.584+0000 7f01218b8640 1 --2- 172.21.15.92:0/3962024205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011c100740 0x7f011c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.584+0000 7f01218b8640 1 -- 172.21.15.92:0/3962024205 >> 172.21.15.92:0/3962024205 conn(0x7f011c0fbcb0 msgr2=0x7f011c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.584+0000 7f01218b8640 1 -- 172.21.15.92:0/3962024205 shutdown_connections 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.584+0000 7f01218b8640 1 -- 172.21.15.92:0/3962024205 wait complete. 2024-03-19T02:00:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f01218b8640 1 Processor -- start 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f01218b8640 1 -- start start 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f01218b8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011c100740 0x7f011c1046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f01218b8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c101930 0x7f011c102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f01218b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011c1022f0 0x7f011c103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f01218b8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f011c10c3c0 con 0x7f011c101930 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f01218b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f011c10c240 con 0x7f011c1022f0 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f01218b8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f011c10c540 con 0x7f011c100740 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f0113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c101930 0x7f011c102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f011affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011c100740 0x7f011c1046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f011affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011c100740 0x7f011c1046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:56564/0 (socket says 172.21.15.92:56564) 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f0113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c101930 0x7f011c102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37384/0 (socket says 172.21.15.92:37384) 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f011affd640 1 -- 172.21.15.92:0/853745882 learned_addr learned my addr 172.21.15.92:0/853745882 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f011b7fe640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011c1022f0 0x7f011c103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f011affd640 1 -- 172.21.15.92:0/853745882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011c1022f0 msgr2=0x7f011c103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f011affd640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011c1022f0 0x7f011c103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f011affd640 1 -- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c101930 msgr2=0x7f011c102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.585+0000 7f011affd640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c101930 0x7f011c102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.586+0000 7f011affd640 1 -- 172.21.15.92:0/853745882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011c103ae0 con 0x7f011c100740 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.586+0000 7f0113fff640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c101930 0x7f011c102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.586+0000 7f011affd640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011c100740 0x7f011c1046d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e9c0 tx=0x7f010c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.586+0000 7f01208b6640 1 -- 172.21.15.92:0/853745882 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00cd80 con 0x7f011c100740 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.586+0000 7f01218b8640 1 -- 172.21.15.92:0/853745882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011c1a1b60 con 0x7f011c100740 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.586+0000 7f01208b6640 1 -- 172.21.15.92:0/853745882 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c0035f0 con 0x7f011c100740 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.586+0000 7f01218b8640 1 -- 172.21.15.92:0/853745882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011c1a2060 con 0x7f011c100740 2024-03-19T02:00:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.586+0000 7f01208b6640 1 -- 172.21.15.92:0/853745882 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c010630 con 0x7f011c100740 2024-03-19T02:00:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.588+0000 7f01208b6640 1 -- 172.21.15.92:0/853745882 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f010c0037a0 con 0x7f011c100740 2024-03-19T02:00:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.588+0000 7f0124578740 1 Processor -- start 2024-03-19T02:00:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f0124578740 1 -- start start 2024-03-19T02:00:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f0124578740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b49102a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f0124578740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102aaf0 0x55b49102f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f0124578740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 0x55b4910320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f0124578740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b490f8ebe0 con 0x55b49102a100 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f0124578740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b490f37960 con 0x55b49102aaf0 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f0124578740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b490f8ac80 con 0x55b49102fc60 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b49102a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b49102a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37400/0 (socket says 172.21.15.92:37400) 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 -- 172.21.15.92:0/1778353082 learned_addr learned my addr 172.21.15.92:0/1778353082 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f0113fff640 1 --2- 172.21.15.92:0/1778353082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 0x55b4910320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011affd640 1 --2- 172.21.15.92:0/1778353082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102aaf0 0x55b49102f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f01208b6640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f01080a28a0 0x7f01080a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 -- 172.21.15.92:0/1778353082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 msgr2=0x55b4910320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 --2- 172.21.15.92:0/1778353082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 0x55b4910320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 -- 172.21.15.92:0/1778353082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102aaf0 msgr2=0x55b49102f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 --2- 172.21.15.92:0/1778353082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102aaf0 0x55b49102f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 -- 172.21.15.92:0/1778353082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b490f2ed20 con 0x55b49102a100 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f01080a28a0 0x7f01080a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f01208b6640 1 -- 172.21.15.92:0/853745882 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f010c0cef40 con 0x7f011c100740 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.589+0000 7f011b7fe640 1 --2- 172.21.15.92:0/1778353082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b49102a520 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0114009ca0 tx=0x7f0114005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8132aaf94625270b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0111ffb640 1 -- 172.21.15.92:0/1778353082 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114039040 con 0x55b49102a100 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0111ffb640 1 -- 172.21.15.92:0/1778353082 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114003e50 con 0x55b49102a100 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0111ffb640 1 -- 172.21.15.92:0/1778353082 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011404a440 con 0x55b49102a100 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0124578740 1 -- 172.21.15.92:0/1778353082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 msgr2=0x55b49102a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f011b7fe640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f01080a28a0 0x7f01080a4d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f011404c000 tx=0x7f0114045000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0124578740 1 --2- 172.21.15.92:0/1778353082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b49102a520 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0114009ca0 tx=0x7f0114005f70 comp rx=0 tx=0).stop 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0124578740 1 -- 172.21.15.92:0/1778353082 shutdown_connections 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0124578740 1 --2- 172.21.15.92:0/1778353082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 0x55b4910320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0124578740 1 --2- 172.21.15.92:0/1778353082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102aaf0 0x55b49102f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0124578740 1 --2- 172.21.15.92:0/1778353082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b49102a520 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0124578740 1 -- 172.21.15.92:0/1778353082 >> 172.21.15.92:0/1778353082 conn(0x55b491025670 msgr2=0x55b491025a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0124578740 1 -- 172.21.15.92:0/1778353082 shutdown_connections 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.590+0000 7f0124578740 1 -- 172.21.15.92:0/1778353082 wait complete. 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0124578740 1 -- start start 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0124578740 1 Processor -- start 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0124578740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 0x55b4910340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0124578740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102aaf0 0x55b4910345e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0124578740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 0x55b49103b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0124578740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b491032c10 con 0x55b49102aaf0 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0124578740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b490f4adb0 con 0x55b49102a100 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0124578740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b491032d90 con 0x55b49102fc60 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0124578740 10 client.0 _ll_register_callbacks cb 0x55b490f4fe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:00:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102aaf0 0x55b4910345e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 0x55b49103b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f011affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 0x55b4910340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0113fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102aaf0 0x55b4910345e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37414/0 (socket says 172.21.15.92:37414) 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0113fff640 1 -- 172.21.15.92:0/2506243379 learned_addr learned my addr 172.21.15.92:0/2506243379 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0113fff640 1 -- 172.21.15.92:0/2506243379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 msgr2=0x55b49103b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0113fff640 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 0x55b49103b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0113fff640 1 -- 172.21.15.92:0/2506243379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 msgr2=0x55b4910340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0113fff640 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 0x55b4910340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f0113fff640 1 -- 172.21.15.92:0/2506243379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b490f49c60 con 0x55b49102aaf0 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.591+0000 7f011affd640 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 0x55b4910340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0113fff640 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102aaf0 0x55b4910345e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0104012b10 tx=0x7f010400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b90bfa749ebe3e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f00faffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f00faffd640 1 -- 172.21.15.92:0/2506243379 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010400fae0 con 0x55b49102aaf0 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f00faffd640 1 -- 172.21.15.92:0/2506243379 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f010400fc80 con 0x55b49102aaf0 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0124578740 1 -- 172.21.15.92:0/2506243379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0114003e50 con 0x55b49102aaf0 2024-03-19T02:00:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f00faffd640 1 -- 172.21.15.92:0/2506243379 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0104004050 con 0x55b49102aaf0 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0124578740 10 client.15942 fetch_fsmap learned FSMap version 29 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0124578740 1 -- 172.21.15.92:0/2506243379 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b490ca3210 con 0x55b49102aaf0 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f00faffd640 1 -- 172.21.15.92:0/2506243379 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114003e50 con 0x55b49102aaf0 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f00faffd640 1 -- 172.21.15.92:0/2506243379 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0104010430 con 0x55b49102aaf0 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0124578740 10 client.15942 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0124578740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0124578740 20 client.15942 populate_metadata read hostname 'smithi092' 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0124578740 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b49103ff90 0x55b491042450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0124578740 4 client.15942 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.592+0000 7f0124578740 1 -- 172.21.15.92:0/2506243379 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b490f9c120 con 0x55b49103ff90 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.593+0000 7f011affd640 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b49103ff90 0x55b491042450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.593+0000 7f00faffd640 10 client.15942 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.593+0000 7f011affd640 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b49103ff90 0x55b491042450 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3267b505a5008a server_cookie=906bfa0f802e96aa in_seq=0 out_seq=0 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f00faffd640 1 -- 172.21.15.92:0/2506243379 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b490f9c120 con 0x55b49103ff90 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f00faffd640 10 client.15942 handle_command_reply: tid=0 2024-03-19T02:00:34.601 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 client.15942 shutdown 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 -- 172.21.15.92:0/2506243379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102aaf0 msgr2=0x55b4910345e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102aaf0 0x55b4910345e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0104012b10 tx=0x7f010400f3c0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 -- 172.21.15.92:0/2506243379 shutdown_connections 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b49103ff90 0x55b491042450 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102fc60 0x55b49103b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f00faffd640 0 client.15942 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102aaf0 0x55b4910345e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 --2- 172.21.15.92:0/2506243379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 0x55b4910340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 -- 172.21.15.92:0/2506243379 >> 172.21.15.92:0/2506243379 conn(0x55b491025670 msgr2=0x55b4910268c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 -- 172.21.15.92:0/2506243379 shutdown_connections 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 1 -- 172.21.15.92:0/2506243379 wait complete. 2024-03-19T02:00:34.602 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.594+0000 7f0124578740 20 client.15942 trim_cache size 0 max 16384 2024-03-19T02:00:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.608+0000 7f0124578740 1 Processor -- start 2024-03-19T02:00:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.608+0000 7f0124578740 1 -- start start 2024-03-19T02:00:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f0124578740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 0x55b491024220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f0124578740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 0x55b49103b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f0124578740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102fc60 0x55b491024760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f0124578740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1aae20 con 0x55b49102fc60 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f0124578740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1ac960 con 0x55b49102a100 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f0124578740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1ab4f0 con 0x55b49102aaf0 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102fc60 0x55b491024760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102fc60 0x55b491024760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37416/0 (socket says 172.21.15.92:37416) 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011b7fe640 1 -- 172.21.15.92:0/2836528792 learned_addr learned my addr 172.21.15.92:0/2836528792 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f0113fff640 1 --2- 172.21.15.92:0/2836528792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 0x55b49103b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011affd640 1 --2- 172.21.15.92:0/2836528792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 0x55b491024220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011b7fe640 1 -- 172.21.15.92:0/2836528792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 msgr2=0x55b49103b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011b7fe640 1 --2- 172.21.15.92:0/2836528792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 0x55b49103b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011b7fe640 1 -- 172.21.15.92:0/2836528792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 msgr2=0x55b491024220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011b7fe640 1 --2- 172.21.15.92:0/2836528792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 0x55b491024220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011b7fe640 1 -- 172.21.15.92:0/2836528792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b490d6f620 con 0x55b49102fc60 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f011b7fe640 1 --2- 172.21.15.92:0/2836528792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102fc60 0x55b491024760 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f011404d040 tx=0x7f0114052000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56181c59b97b9eed server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f01117fa640 1 -- 172.21.15.92:0/2836528792 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114039050 con 0x55b49102fc60 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f01117fa640 1 -- 172.21.15.92:0/2836528792 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011405c070 con 0x55b49102fc60 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.609+0000 7f01117fa640 1 -- 172.21.15.92:0/2836528792 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011404ad50 con 0x55b49102fc60 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 -- 172.21.15.92:0/2836528792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102fc60 msgr2=0x55b491024760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 --2- 172.21.15.92:0/2836528792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102fc60 0x55b491024760 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f011404d040 tx=0x7f0114052000 comp rx=0 tx=0).stop 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 -- 172.21.15.92:0/2836528792 shutdown_connections 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 --2- 172.21.15.92:0/2836528792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102fc60 0x55b491024760 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 --2- 172.21.15.92:0/2836528792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 0x55b49103b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 --2- 172.21.15.92:0/2836528792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102a100 0x55b491024220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 -- 172.21.15.92:0/2836528792 >> 172.21.15.92:0/2836528792 conn(0x55b491025670 msgr2=0x55b491025a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 -- 172.21.15.92:0/2836528792 shutdown_connections 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 -- 172.21.15.92:0/2836528792 wait complete. 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.610+0000 7f0124578740 1 -- start start 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0124578740 1 Processor -- start 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0124578740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b491038c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0124578740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 0x55b4910391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0124578740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102fc60 0x55b49103a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0124578740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1ac290 con 0x55b49102a100 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0124578740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1abbc0 con 0x55b49102fc60 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0124578740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f011c1a8e00 con 0x55b49102aaf0 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0124578740 10 client.0 _ll_register_callbacks cb 0x7f0104009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b491038c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b491038c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37432/0 (socket says 172.21.15.92:37432) 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011affd640 1 -- 172.21.15.92:0/3661380665 learned_addr learned my addr 172.21.15.92:0/3661380665 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0113fff640 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 0x55b4910391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011b7fe640 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102fc60 0x55b49103a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011affd640 1 -- 172.21.15.92:0/3661380665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 msgr2=0x55b4910391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011affd640 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 0x55b4910391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011affd640 1 -- 172.21.15.92:0/3661380665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102fc60 msgr2=0x55b49103a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011affd640 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102fc60 0x55b49103a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011affd640 1 -- 172.21.15.92:0/3661380665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49103a770 con 0x55b49102a100 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f011affd640 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b491038c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f010c003ec0 tx=0x7f010c0cf2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09cd2370cea4f5f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f00faffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f00faffd640 1 -- 172.21.15.92:0/3661380665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c09fa40 con 0x55b49102a100 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f00faffd640 1 -- 172.21.15.92:0/3661380665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f010c003b10 con 0x55b49102a100 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.611+0000 7f0124578740 1 -- 172.21.15.92:0/3661380665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f011405c070 con 0x55b49102a100 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f00faffd640 1 -- 172.21.15.92:0/3661380665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f010c0a4090 con 0x55b49102a100 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0124578740 10 client.15948 fetch_fsmap learned FSMap version 29 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0124578740 1 -- 172.21.15.92:0/3661380665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4910a6350 con 0x55b49102a100 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f00faffd640 1 -- 172.21.15.92:0/3661380665 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011405c070 con 0x55b49102a100 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f00faffd640 1 -- 172.21.15.92:0/3661380665 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f010c0d2040 con 0x55b49102a100 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0124578740 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0124578740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0124578740 20 client.15948 populate_metadata read hostname 'smithi092' 2024-03-19T02:00:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0124578740 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b4910a6ce0 0x55b4910a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0124578740 4 client.15948 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:00:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0124578740 1 -- 172.21.15.92:0/3661380665 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b490f96470 con 0x55b4910a6ce0 2024-03-19T02:00:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0113fff640 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b4910a6ce0 0x55b4910a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f0113fff640 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b4910a6ce0 0x55b4910a90d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=486062ae3e5ce122 server_cookie=bd2d07ddba0daf69 in_seq=0 out_seq=0 2024-03-19T02:00:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.612+0000 7f00faffd640 10 client.15948 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f00faffd640 1 -- 172.21.15.92:0/3661380665 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b490f96470 con 0x55b4910a6ce0 2024-03-19T02:00:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f00faffd640 10 client.15948 handle_command_reply: tid=0 2024-03-19T02:00:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 client.15948 shutdown 2024-03-19T02:00:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 -- 172.21.15.92:0/3661380665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 msgr2=0x55b491038c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b491038c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f010c003ec0 tx=0x7f010c0cf2d0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 -- 172.21.15.92:0/3661380665 shutdown_connections 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55b4910a6ce0 0x55b4910a90d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b49102fc60 0x55b49103a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b49102aaf0 0x55b4910391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f00faffd640 0 client.15948 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 --2- 172.21.15.92:0/3661380665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b49102a100 0x55b491038c80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 -- 172.21.15.92:0/3661380665 >> 172.21.15.92:0/3661380665 conn(0x55b491025670 msgr2=0x55b491034060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 -- 172.21.15.92:0/3661380665 shutdown_connections 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.619+0000 7f0124578740 1 -- 172.21.15.92:0/3661380665 wait complete. 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f0124578740 20 client.15948 trim_cache size 0 max 16384 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:00:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:00:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 -- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f01080a28a0 msgr2=0x7f01080a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f01080a28a0 0x7f01080a4d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f011404c000 tx=0x7f0114045000 comp rx=0 tx=0).stop 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 -- 172.21.15.92:0/853745882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011c100740 msgr2=0x7f011c1046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011c100740 0x7f011c1046d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e9c0 tx=0x7f010c00ee80 comp rx=0 tx=0).stop 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 -- 172.21.15.92:0/853745882 shutdown_connections 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011c1022f0 0x7f011c103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f011c101930 0x7f011c102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f01080a28a0 0x7f01080a4d60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 --2- 172.21.15.92:0/853745882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f011c100740 0x7f011c1046d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 -- 172.21.15.92:0/853745882 >> 172.21.15.92:0/853745882 conn(0x7f011c0fbcb0 msgr2=0x7f011c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 -- 172.21.15.92:0/853745882 shutdown_connections 2024-03-19T02:00:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:34.620+0000 7f01218b8640 1 -- 172.21.15.92:0/853745882 wait complete. 2024-03-19T02:00:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:34 smithi159 ceph-mon[67110]: pgmap v584: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:00:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:34 smithi092 ceph-mon[62574]: pgmap v584: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:00:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:34 smithi144 ceph-mon[67199]: pgmap v584: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:00:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:36 smithi159 ceph-mon[67110]: pgmap v585: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-03-19T02:00:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:36 smithi092 ceph-mon[62574]: pgmap v585: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-03-19T02:00:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:36 smithi144 ceph-mon[67199]: pgmap v585: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-03-19T02:00:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:38 smithi159 ceph-mon[67110]: pgmap v586: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T02:00:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:38 smithi092 ceph-mon[62574]: pgmap v586: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T02:00:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:38 smithi144 ceph-mon[67199]: pgmap v586: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T02:00:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:40 smithi159 ceph-mon[67110]: pgmap v587: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:40 smithi092 ceph-mon[62574]: pgmap v587: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:40 smithi144 ceph-mon[67199]: pgmap v587: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:42 smithi159 ceph-mon[67110]: pgmap v588: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:00:43.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:42 smithi092 ceph-mon[62574]: pgmap v588: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:00:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:42 smithi144 ceph-mon[67199]: pgmap v588: 97 pgs: 97 active+clean; 2.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:00:44.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:44 smithi159 ceph-mon[67110]: pgmap v589: 97 pgs: 97 active+clean; 2.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:00:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:44 smithi092 ceph-mon[62574]: pgmap v589: 97 pgs: 97 active+clean; 2.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:00:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:44 smithi144 ceph-mon[67199]: pgmap v589: 97 pgs: 97 active+clean; 2.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:00:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:46 smithi159 ceph-mon[67110]: pgmap v590: 97 pgs: 97 active+clean; 2.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:00:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:46 smithi092 ceph-mon[62574]: pgmap v590: 97 pgs: 97 active+clean; 2.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:00:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:46 smithi144 ceph-mon[67199]: pgmap v590: 97 pgs: 97 active+clean; 2.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:00:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 -- 172.21.15.92:0/4100479197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc103420 msgr2=0x7fa6fc109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 --2- 172.21.15.92:0/4100479197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc103420 0x7fa6fc109cb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009ca0 tx=0x7fa6f402c0f0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 -- 172.21.15.92:0/4100479197 shutdown_connections 2024-03-19T02:00:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 --2- 172.21.15.92:0/4100479197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc103420 0x7fa6fc109cb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 --2- 172.21.15.92:0/4100479197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc102a60 0x7fa6fc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 --2- 172.21.15.92:0/4100479197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6fc101870 0x7fa6fc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 -- 172.21.15.92:0/4100479197 >> 172.21.15.92:0/4100479197 conn(0x7fa6fc0fcde0 msgr2=0x7fa6fc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 -- 172.21.15.92:0/4100479197 shutdown_connections 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 -- 172.21.15.92:0/4100479197 wait complete. 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 Processor -- start 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.714+0000 7fa702f76640 1 -- start start 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa702f76640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc101870 0x7fa6fc199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa702f76640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6fc102a60 0x7fa6fc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa702f76640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc103420 0x7fa6fc1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa702f76640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc10d4d0 con 0x7fa6fc101870 2024-03-19T02:00:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa702f76640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc10d350 con 0x7fa6fc102a60 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa702f76640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc10d650 con 0x7fa6fc103420 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa700d12640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc101870 0x7fa6fc199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa700d12640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc101870 0x7fa6fc199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58700/0 (socket says 172.21.15.92:58700) 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa700d12640 1 -- 172.21.15.92:0/1580445447 learned_addr learned my addr 172.21.15.92:0/1580445447 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6fc102a60 0x7fa6fc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa701513640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc103420 0x7fa6fc1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa700d12640 1 -- 172.21.15.92:0/1580445447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc103420 msgr2=0x7fa6fc1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa700d12640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc103420 0x7fa6fc1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa700d12640 1 -- 172.21.15.92:0/1580445447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6fc102a60 msgr2=0x7fa6fc19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa700d12640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6fc102a60 0x7fa6fc19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa700d12640 1 -- 172.21.15.92:0/1580445447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc1a1940 con 0x7fa6fc101870 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa700d12640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc101870 0x7fa6fc199d30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000e9c0 tx=0x7fa6f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa6f9ffb640 1 -- 172.21.15.92:0/1580445447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000cd80 con 0x7fa6fc101870 2024-03-19T02:00:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa6f9ffb640 1 -- 172.21.15.92:0/1580445447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f00035f0 con 0x7fa6fc101870 2024-03-19T02:00:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa702f76640 1 -- 172.21.15.92:0/1580445447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc1a1c30 con 0x7fa6fc101870 2024-03-19T02:00:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.715+0000 7fa6f9ffb640 1 -- 172.21.15.92:0/1580445447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0010690 con 0x7fa6fc101870 2024-03-19T02:00:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.716+0000 7fa702f76640 1 -- 172.21.15.92:0/1580445447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc1a2180 con 0x7fa6fc101870 2024-03-19T02:00:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa705c35740 1 Processor -- start 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa705c35740 1 -- start start 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa6f9ffb640 1 -- 172.21.15.92:0/1580445447 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fa6f00037a0 con 0x7fa6fc101870 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa705c35740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f702520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa705c35740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f702af0 0x55bd8f7075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa705c35740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f707c60 0x55bd8f70a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa705c35740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f666be0 con 0x55bd8f707c60 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa705c35740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f60f960 con 0x55bd8f702af0 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa705c35740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f662c80 con 0x55bd8f702100 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f707c60 0x55bd8f70a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa701513640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f702520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa701513640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f702520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:45834/0 (socket says 172.21.15.92:45834) 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f707c60 0x55bd8f70a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58706/0 (socket says 172.21.15.92:58706) 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa701513640 1 -- 172.21.15.92:0/1459159260 learned_addr learned my addr 172.21.15.92:0/1459159260 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.718+0000 7fa700d12640 1 --2- 172.21.15.92:0/1459159260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f702af0 0x55bd8f7075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa701513640 1 -- 172.21.15.92:0/1459159260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f702af0 msgr2=0x55bd8f7075f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa701513640 1 --2- 172.21.15.92:0/1459159260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f702af0 0x55bd8f7075f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa701513640 1 -- 172.21.15.92:0/1459159260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f707c60 msgr2=0x55bd8f70a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa701513640 1 --2- 172.21.15.92:0/1459159260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f707c60 0x55bd8f70a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa701513640 1 -- 172.21.15.92:0/1459159260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd8f606d20 con 0x55bd8f702100 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/1459159260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f707c60 0x55bd8f70a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa6f9ffb640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa6ec0a28a0 0x7fa6ec0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa700d12640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa6ec0a28a0 0x7fa6ec0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa701513640 1 --2- 172.21.15.92:0/1459159260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f702520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009ca0 tx=0x7fa6f40323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd4a97a923465d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa6f9ffb640 1 -- 172.21.15.92:0/1580445447 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa6f00cee90 con 0x7fa6fc101870 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa6e6ffd640 1 -- 172.21.15.92:0/1459159260 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4043070 con 0x55bd8f702100 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa6e6ffd640 1 -- 172.21.15.92:0/1459159260 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f4032c90 con 0x55bd8f702100 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.719+0000 7fa6e6ffd640 1 -- 172.21.15.92:0/1459159260 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4039040 con 0x55bd8f702100 2024-03-19T02:00:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 -- 172.21.15.92:0/1459159260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 msgr2=0x55bd8f702520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa700d12640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa6ec0a28a0 0x7fa6ec0a4d60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00ccd10 tx=0x7fa6f00d2970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 --2- 172.21.15.92:0/1459159260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f702520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009ca0 tx=0x7fa6f40323c0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 -- 172.21.15.92:0/1459159260 shutdown_connections 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 --2- 172.21.15.92:0/1459159260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f707c60 0x55bd8f70a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 --2- 172.21.15.92:0/1459159260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f702af0 0x55bd8f7075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 --2- 172.21.15.92:0/1459159260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f702520 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 -- 172.21.15.92:0/1459159260 >> 172.21.15.92:0/1459159260 conn(0x55bd8f6fd670 msgr2=0x55bd8f6fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 -- 172.21.15.92:0/1459159260 shutdown_connections 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 -- 172.21.15.92:0/1459159260 wait complete. 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 -- start start 2024-03-19T02:00:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.720+0000 7fa705c35740 1 Processor -- start 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa705c35740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f70c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa705c35740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f70c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa705c35740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f7135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa705c35740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f70abb0 con 0x55bd8f702100 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa705c35740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f622db0 con 0x55bd8f707c60 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa705c35740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f70ad30 con 0x55bd8f702af0 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa705c35740 10 client.0 _ll_register_callbacks cb 0x55bd8f627e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa701513640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f70c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa701513640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f70c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58714/0 (socket says 172.21.15.92:58714) 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa701513640 1 -- 172.21.15.92:0/2737946340 learned_addr learned my addr 172.21.15.92:0/2737946340 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f70c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa701513640 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f7135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6fbfff640 1 -- 172.21.15.92:0/2737946340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 msgr2=0x55bd8f7135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f7135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6fbfff640 1 -- 172.21.15.92:0/2737946340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 msgr2=0x55bd8f70c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f70c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6fbfff640 1 -- 172.21.15.92:0/2737946340 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd8f621c60 con 0x55bd8f702af0 2024-03-19T02:00:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa701513640 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f70c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:00:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f70c620 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80126d0 tx=0x7fa6e8012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aee1ced81a3389c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-19T02:00:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6dbfff640 1 -- 172.21.15.92:0/2737946340 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800f8c0 con 0x55bd8f702af0 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa6dbfff640 1 -- 172.21.15.92:0/2737946340 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa6e800fa60 con 0x55bd8f702af0 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.721+0000 7fa705c35740 1 -- 172.21.15.92:0/2737946340 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f4032c90 con 0x55bd8f702af0 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa6dbfff640 1 -- 172.21.15.92:0/2737946340 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6e800fc60 con 0x55bd8f702af0 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa705c35740 10 client.25846 fetch_fsmap learned FSMap version 29 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa705c35740 1 -- 172.21.15.92:0/2737946340 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd8f37b210 con 0x55bd8f702af0 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa6dbfff640 1 -- 172.21.15.92:0/2737946340 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f4032c90 con 0x55bd8f702af0 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa6dbfff640 1 -- 172.21.15.92:0/2737946340 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa6e8010430 con 0x55bd8f702af0 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa705c35740 10 client.25846 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa705c35740 10 client.25846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa705c35740 20 client.25846 populate_metadata read hostname 'smithi092' 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa705c35740 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bd8f717f60 0x55bd8f71a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa705c35740 4 client.25846 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa705c35740 1 -- 172.21.15.92:0/2737946340 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd8f674120 con 0x55bd8f717f60 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.722+0000 7fa701513640 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bd8f717f60 0x55bd8f71a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.723+0000 7fa701513640 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bd8f717f60 0x55bd8f71a3a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7408dadb928f5537 server_cookie=8df6c24e4ec4b35b in_seq=0 out_seq=0 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.723+0000 7fa6dbfff640 10 client.25846 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.723+0000 7fa6dbfff640 1 -- 172.21.15.92:0/2737946340 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bd8f674120 con 0x55bd8f717f60 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.723+0000 7fa6dbfff640 10 client.25846 handle_command_reply: tid=0 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.723+0000 7fa705c35740 1 client.25846 shutdown 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 -- 172.21.15.92:0/2737946340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 msgr2=0x55bd8f70c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f70c620 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80126d0 tx=0x7fa6e8012b90 comp rx=0 tx=0).stop 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 -- 172.21.15.92:0/2737946340 shutdown_connections 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bd8f717f60 0x55bd8f71a3a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f7135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f70c620 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 --2- 172.21.15.92:0/2737946340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f70c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa6dbfff640 0 client.25846 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 -- 172.21.15.92:0/2737946340 >> 172.21.15.92:0/2737946340 conn(0x55bd8f6fd670 msgr2=0x55bd8f6fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 -- 172.21.15.92:0/2737946340 shutdown_connections 2024-03-19T02:00:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 1 -- 172.21.15.92:0/2737946340 wait complete. 2024-03-19T02:00:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.724+0000 7fa705c35740 20 client.25846 trim_cache size 0 max 16384 2024-03-19T02:00:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.746+0000 7fa705c35740 1 Processor -- start 2024-03-19T02:00:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.746+0000 7fa705c35740 1 -- start start 2024-03-19T02:00:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa705c35740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f6fc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa705c35740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f7135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa705c35740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f703590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa705c35740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1aaed0 con 0x55bd8f702100 2024-03-19T02:00:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa705c35740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1aca10 con 0x55bd8f707c60 2024-03-19T02:00:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa705c35740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1ab5a0 con 0x55bd8f702af0 2024-03-19T02:00:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f6fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f6fc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58728/0 (socket says 172.21.15.92:58728) 2024-03-19T02:00:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 -- 172.21.15.92:0/3324210982 learned_addr learned my addr 172.21.15.92:0/3324210982 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f7135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f703590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 -- 172.21.15.92:0/3324210982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 msgr2=0x55bd8f7135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f7135b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 -- 172.21.15.92:0/3324210982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 msgr2=0x55bd8f703590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f703590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 -- 172.21.15.92:0/3324210982 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd8f447620 con 0x55bd8f702100 2024-03-19T02:00:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f7135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:00:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa701513640 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f6fc930 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005ca0 tx=0x7fa6f4032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e83009d1b5c69f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa6e67fc640 1 -- 172.21.15.92:0/3324210982 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4039050 con 0x55bd8f702100 2024-03-19T02:00:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa6e67fc640 1 -- 172.21.15.92:0/3324210982 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f4032c90 con 0x55bd8f702100 2024-03-19T02:00:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.747+0000 7fa6e67fc640 1 -- 172.21.15.92:0/3324210982 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4043040 con 0x55bd8f702100 2024-03-19T02:00:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 -- 172.21.15.92:0/3324210982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 msgr2=0x55bd8f6fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f6fc930 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005ca0 tx=0x7fa6f4032590 comp rx=0 tx=0).stop 2024-03-19T02:00:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 -- 172.21.15.92:0/3324210982 shutdown_connections 2024-03-19T02:00:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f703590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702af0 0x55bd8f7135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 --2- 172.21.15.92:0/3324210982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702100 0x55bd8f6fc930 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 -- 172.21.15.92:0/3324210982 >> 172.21.15.92:0/3324210982 conn(0x55bd8f6fd670 msgr2=0x55bd8f6fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 -- 172.21.15.92:0/3324210982 shutdown_connections 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 -- 172.21.15.92:0/3324210982 wait complete. 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 -- start start 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.748+0000 7fa705c35740 1 Processor -- start 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa705c35740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f710530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa705c35740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702af0 0x55bd8f710a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa705c35740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f77c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa705c35740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1ac340 con 0x55bd8f702af0 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa705c35740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1abc70 con 0x55bd8f707c60 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa705c35740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1a8eb0 con 0x55bd8f702100 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa705c35740 10 client.0 _ll_register_callbacks cb 0x7fa6e8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:00:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702af0 0x55bd8f710a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702af0 0x55bd8f710a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58744/0 (socket says 172.21.15.92:58744) 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6fbfff640 1 -- 172.21.15.92:0/1785765443 learned_addr learned my addr 172.21.15.92:0/1785765443 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa701513640 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f77c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa701513640 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f710530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6fbfff640 1 -- 172.21.15.92:0/1785765443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 msgr2=0x55bd8f710530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f710530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6fbfff640 1 -- 172.21.15.92:0/1785765443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 msgr2=0x55bd8f77c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f77c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6fbfff640 1 -- 172.21.15.92:0/1785765443 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd8f77c800 con 0x55bd8f702af0 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa701513640 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f710530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6fbfff640 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702af0 0x55bd8f710a70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000c00 tx=0x7fa6e800ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32de6254009a1fc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:00:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.749+0000 7fa6dbfff640 1 -- 172.21.15.92:0/1785765443 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8010430 con 0x55bd8f702af0 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa6dbfff640 1 -- 172.21.15.92:0/1785765443 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa6e80105d0 con 0x55bd8f702af0 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa705c35740 1 -- 172.21.15.92:0/1785765443 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f4032c90 con 0x55bd8f702af0 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa6dbfff640 1 -- 172.21.15.92:0/1785765443 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6e800f790 con 0x55bd8f702af0 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa705c35740 10 client.15966 fetch_fsmap learned FSMap version 29 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa705c35740 1 -- 172.21.15.92:0/1785765443 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd8f77daf0 con 0x55bd8f702af0 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa6dbfff640 1 -- 172.21.15.92:0/1785765443 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f4032c90 con 0x55bd8f702af0 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa6dbfff640 1 -- 172.21.15.92:0/1785765443 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa6e802f430 con 0x55bd8f702af0 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa705c35740 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa705c35740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa705c35740 20 client.15966 populate_metadata read hostname 'smithi092' 2024-03-19T02:00:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa705c35740 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bd8f77e470 0x55bd8f780890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:00:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa705c35740 4 client.15966 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:00:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa705c35740 1 -- 172.21.15.92:0/1785765443 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd8f66e470 con 0x55bd8f77e470 2024-03-19T02:00:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.750+0000 7fa701513640 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bd8f77e470 0x55bd8f780890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:00:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.751+0000 7fa6dbfff640 10 client.15966 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.751+0000 7fa701513640 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bd8f77e470 0x55bd8f780890 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9609e3d16e727481 server_cookie=f49c97b6d6ea4cf2 in_seq=0 out_seq=0 2024-03-19T02:00:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.757+0000 7fa6dbfff640 1 -- 172.21.15.92:0/1785765443 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd8f66e470 con 0x55bd8f77e470 2024-03-19T02:00:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.757+0000 7fa6dbfff640 10 client.15966 handle_command_reply: tid=0 2024-03-19T02:00:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.757+0000 7fa705c35740 1 client.15966 shutdown 2024-03-19T02:00:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 -- 172.21.15.92:0/1785765443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702af0 msgr2=0x55bd8f710a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702af0 0x55bd8f710a70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000c00 tx=0x7fa6e800ffb0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 -- 172.21.15.92:0/1785765443 shutdown_connections 2024-03-19T02:00:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bd8f77e470 0x55bd8f780890 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bd8f707c60 0x55bd8f77c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd8f702af0 0x55bd8f710a70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa6dbfff640 0 client.15966 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:00:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 --2- 172.21.15.92:0/1785765443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f702100 0x55bd8f710530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 -- 172.21.15.92:0/1785765443 >> 172.21.15.92:0/1785765443 conn(0x55bd8f6fd670 msgr2=0x55bd8f7008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 -- 172.21.15.92:0/1785765443 shutdown_connections 2024-03-19T02:00:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 1 -- 172.21.15.92:0/1785765443 wait complete. 2024-03-19T02:00:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa705c35740 20 client.15966 trim_cache size 0 max 16384 2024-03-19T02:00:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:00:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:00:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:00:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 -- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa6ec0a28a0 msgr2=0x7fa6ec0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa6ec0a28a0 0x7fa6ec0a4d60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00ccd10 tx=0x7fa6f00d2970 comp rx=0 tx=0).stop 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 -- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc101870 msgr2=0x7fa6fc199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc101870 0x7fa6fc199d30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000e9c0 tx=0x7fa6f000ee80 comp rx=0 tx=0).stop 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 -- 172.21.15.92:0/1580445447 shutdown_connections 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fa6ec0a28a0 0x7fa6ec0a4d60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc103420 0x7fa6fc1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6fc102a60 0x7fa6fc19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 --2- 172.21.15.92:0/1580445447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc101870 0x7fa6fc199d30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:00:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.758+0000 7fa702f76640 1 -- 172.21.15.92:0/1580445447 >> 172.21.15.92:0/1580445447 conn(0x7fa6fc0fcde0 msgr2=0x7fa6fc073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:00:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.759+0000 7fa702f76640 1 -- 172.21.15.92:0/1580445447 shutdown_connections 2024-03-19T02:00:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:00:47.759+0000 7fa702f76640 1 -- 172.21.15.92:0/1580445447 wait complete. 2024-03-19T02:00:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:48 smithi159 ceph-mon[67110]: pgmap v591: 97 pgs: 97 active+clean; 2.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:00:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:48 smithi092 ceph-mon[62574]: pgmap v591: 97 pgs: 97 active+clean; 2.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:00:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:48 smithi144 ceph-mon[67199]: pgmap v591: 97 pgs: 97 active+clean; 2.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:00:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:50 smithi159 ceph-mon[67110]: pgmap v592: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T02:00:51.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:50 smithi092 ceph-mon[62574]: pgmap v592: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T02:00:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:50 smithi144 ceph-mon[67199]: pgmap v592: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T02:00:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:52 smithi159 ceph-mon[67110]: pgmap v593: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T02:00:53.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:52 smithi092 ceph-mon[62574]: pgmap v593: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T02:00:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:52 smithi144 ceph-mon[67199]: pgmap v593: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-03-19T02:00:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:54 smithi159 ceph-mon[67110]: pgmap v594: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:55.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:54 smithi092 ceph-mon[62574]: pgmap v594: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:55.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:54 smithi144 ceph-mon[67199]: pgmap v594: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:00:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:56 smithi159 ceph-mon[67110]: pgmap v595: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:00:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:56 smithi092 ceph-mon[62574]: pgmap v595: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:00:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:56 smithi144 ceph-mon[67199]: pgmap v595: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:00:57.631 INFO:tasks.workunit.client.0.smithi092.stdout:375/600: open fd = 3 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:376/600: open fd = 3 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:377/600: open fd = 3 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:378/600: open fd = 3 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.632 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:379/600: open fd = 3 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:380/600: open fd = 3 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:381/600: open fd = 3 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:382/600: open fd = 3 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.633 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:383/600: open fd = 3 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:384/600: open fd = 3 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:385/600: open fd = 3 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.634 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:386/600: open fd = 3 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:387/600: open fd = 3 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:388/600: open fd = 3 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:389/600: open fd = 3 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.635 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:390/600: open fd = 3 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:391/600: open fd = 3 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:392/600: open fd = 3 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:393/600: open fd = 3 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.636 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:394/600: open fd = 3 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:395/600: open fd = 3 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:396/600: open fd = 3 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:397/600: open fd = 3 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.637 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:398/600: open fd = 3 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:399/600: open fd = 3 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:400/600: open fd = 3 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:401/600: open fd = 3 2024-03-19T02:00:57.638 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:402/600: open fd = 3 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:403/600: open fd = 3 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:404/600: open fd = 3 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.639 INFO:tasks.workunit.client.0.smithi092.stdout:405/600: open fd = 3 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:406/600: open fd = 3 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:407/600: open fd = 3 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:408/600: open fd = 3 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.640 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:409/600: open fd = 3 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:410/600: open fd = 3 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:411/600: open fd = 3 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:412/600: open fd = 3 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.641 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:413/600: open fd = 3 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:414/600: open fd = 3 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:415/600: open fd = 3 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.642 INFO:tasks.workunit.client.0.smithi092.stdout:416/600: open fd = 3 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:417/600: open fd = 3 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:418/600: open fd = 3 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:419/600: open fd = 3 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:420/600: open fd = 3 2024-03-19T02:00:57.643 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:421/600: open fd = 3 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:422/600: open fd = 3 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:423/600: open fd = 3 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:424/600: open fd = 3 2024-03-19T02:00:57.644 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:425/600: open fd = 3 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:426/600: open fd = 3 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:427/600: open fd = 3 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.645 INFO:tasks.workunit.client.0.smithi092.stdout:428/600: open fd = 3 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:429/600: open fd = 3 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:430/600: open fd = 3 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:431/600: open fd = 3 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.646 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:432/600: open fd = 3 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:433/600: open fd = 3 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:434/600: open fd = 3 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:435/600: open fd = 3 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.647 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.648 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:57.648 INFO:tasks.workunit.client.0.smithi092.stdout:436/600: open fd = 3 2024-03-19T02:00:57.648 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.648 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:00:57.648 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:00:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:00:58 smithi159 ceph-mon[67110]: pgmap v596: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 16 op/s 2024-03-19T02:00:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:00:58 smithi092 ceph-mon[62574]: pgmap v596: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 16 op/s 2024-03-19T02:00:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:00:58 smithi144 ceph-mon[67199]: pgmap v596: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 16 op/s 2024-03-19T02:01:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 -- 172.21.15.92:0/166712106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004101930 msgr2=0x7f3004101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 --2- 172.21.15.92:0/166712106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004101930 0x7f3004101db0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009900 tx=0x7f2ff002bc30 comp rx=0 tx=0).stop 2024-03-19T02:01:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 -- 172.21.15.92:0/166712106 shutdown_connections 2024-03-19T02:01:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 --2- 172.21.15.92:0/166712106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30041022f0 0x7f3004108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 --2- 172.21.15.92:0/166712106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3004101930 0x7f3004101db0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 --2- 172.21.15.92:0/166712106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3004100740 0x7f3004100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 -- 172.21.15.92:0/166712106 >> 172.21.15.92:0/166712106 conn(0x7f30040fbcb0 msgr2=0x7f30040fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 -- 172.21.15.92:0/166712106 shutdown_connections 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 -- 172.21.15.92:0/166712106 wait complete. 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 Processor -- start 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 -- start start 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3004100740 0x7f3004069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3004101930 0x7f300419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30041022f0 0x7f300419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f300410c3c0 con 0x7f30041022f0 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f300410c240 con 0x7f3004100740 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.847+0000 7f300ad1d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f300410c540 con 0x7f3004101930 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f30092ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30041022f0 0x7f300419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f30092ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30041022f0 0x7f300419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45018/0 (socket says 172.21.15.92:45018) 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f3008ab9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3004100740 0x7f3004069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f30092ba640 1 -- 172.21.15.92:0/2343442073 learned_addr learned my addr 172.21.15.92:0/2343442073 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f3003fff640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3004101930 0x7f300419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f30092ba640 1 -- 172.21.15.92:0/2343442073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3004101930 msgr2=0x7f300419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f30092ba640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3004101930 0x7f300419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f30092ba640 1 -- 172.21.15.92:0/2343442073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3004100740 msgr2=0x7f3004069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f30092ba640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3004100740 0x7f3004069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f30092ba640 1 -- 172.21.15.92:0/2343442073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300419f750 con 0x7f30041022f0 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f3008ab9640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3004100740 0x7f3004069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f30092ba640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30041022f0 0x7f300419ee40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00eb10 tx=0x7f2ffc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f3001ffb640 1 -- 172.21.15.92:0/2343442073 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc004b20 con 0x7f30041022f0 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f3001ffb640 1 -- 172.21.15.92:0/2343442073 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc004cc0 con 0x7f30041022f0 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f300ad1d640 1 -- 172.21.15.92:0/2343442073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300419f9e0 con 0x7f30041022f0 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f3001ffb640 1 -- 172.21.15.92:0/2343442073 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc010760 con 0x7f30041022f0 2024-03-19T02:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.848+0000 7f300ad1d640 1 -- 172.21.15.92:0/2343442073 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300419ff00 con 0x7f30041022f0 2024-03-19T02:01:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.850+0000 7f3001ffb640 1 -- 172.21.15.92:0/2343442073 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f2ffc005020 con 0x7f30041022f0 2024-03-19T02:01:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f300d9e0740 1 Processor -- start 2024-03-19T02:01:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f3001ffb640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ff40a2920 0x7f2ff40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f300d9e0740 1 -- start start 2024-03-19T02:01:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 0x561a8ed6e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed73c60 0x561a8ed760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f3003fff640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ff40a2920 0x7f2ff40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f300d9e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561a8ecd2be0 con 0x561a8ed6eaf0 2024-03-19T02:01:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f300d9e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a8ec7b960 con 0x561a8ed6e100 2024-03-19T02:01:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f300d9e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561a8eccec80 con 0x561a8ed73c60 2024-03-19T02:01:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f3001ffb640 1 -- 172.21.15.92:0/2343442073 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2ffc0cef30 con 0x7f30041022f0 2024-03-19T02:01:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f3003fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f3003fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45034/0 (socket says 172.21.15.92:45034) 2024-03-19T02:01:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f3003fff640 1 -- 172.21.15.92:0/2671579533 learned_addr learned my addr 172.21.15.92:0/2671579533 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f3008ab9640 1 --2- 172.21.15.92:0/2671579533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 0x561a8ed6e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f30092ba640 1 --2- 172.21.15.92:0/2671579533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed73c60 0x561a8ed760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.851+0000 7f3008ab9640 1 -- 172.21.15.92:0/2671579533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed73c60 msgr2=0x561a8ed760f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.852+0000 7f3008ab9640 1 --2- 172.21.15.92:0/2671579533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed73c60 0x561a8ed760f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.852+0000 7f3008ab9640 1 -- 172.21.15.92:0/2671579533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 msgr2=0x561a8ed735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.852+0000 7f3008ab9640 1 --2- 172.21.15.92:0/2671579533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.852+0000 7f3008ab9640 1 -- 172.21.15.92:0/2671579533 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8ec72d20 con 0x561a8ed6e100 2024-03-19T02:01:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.852+0000 7f3003fff640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ff40a2920 0x7f2ff40a4de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0036040 tx=0x7f2ff0005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.852+0000 7f3008ab9640 1 --2- 172.21.15.92:0/2671579533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 0x561a8ed6e520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009fa0 tx=0x7f2ff80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685c4b1ae51e3a19 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.852+0000 7f2feeffd640 1 -- 172.21.15.92:0/2671579533 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8004030 con 0x561a8ed6e100 2024-03-19T02:01:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.852+0000 7f2feeffd640 1 -- 172.21.15.92:0/2671579533 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff80041d0 con 0x561a8ed6e100 2024-03-19T02:01:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.852+0000 7f2feeffd640 1 -- 172.21.15.92:0/2671579533 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff800f870 con 0x561a8ed6e100 2024-03-19T02:01:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 -- 172.21.15.92:0/2671579533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 msgr2=0x561a8ed6e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 --2- 172.21.15.92:0/2671579533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 0x561a8ed6e520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009fa0 tx=0x7f2ff80118d0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 -- 172.21.15.92:0/2671579533 shutdown_connections 2024-03-19T02:01:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 --2- 172.21.15.92:0/2671579533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed73c60 0x561a8ed760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 --2- 172.21.15.92:0/2671579533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 --2- 172.21.15.92:0/2671579533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 0x561a8ed6e520 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 -- 172.21.15.92:0/2671579533 >> 172.21.15.92:0/2671579533 conn(0x561a8ed69670 msgr2=0x561a8ed69a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 -- 172.21.15.92:0/2671579533 shutdown_connections 2024-03-19T02:01:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 -- 172.21.15.92:0/2671579533 wait complete. 2024-03-19T02:01:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 -- start start 2024-03-19T02:01:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 Processor -- start 2024-03-19T02:01:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 0x561a8ed78110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed78650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed73c60 0x561a8ed7f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561a8ed76c10 con 0x561a8ed6eaf0 2024-03-19T02:01:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a8ec8edb0 con 0x561a8ed73c60 2024-03-19T02:01:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.853+0000 7f300d9e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561a8ed76d90 con 0x561a8ed6e100 2024-03-19T02:01:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3003fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed78650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f300d9e0740 10 client.0 _ll_register_callbacks cb 0x561a8ec93e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3003fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed78650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45038/0 (socket says 172.21.15.92:45038) 2024-03-19T02:01:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3003fff640 1 -- 172.21.15.92:0/480278059 learned_addr learned my addr 172.21.15.92:0/480278059 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3008ab9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 0x561a8ed78110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f30092ba640 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed73c60 0x561a8ed7f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3003fff640 1 -- 172.21.15.92:0/480278059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 msgr2=0x561a8ed78110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3003fff640 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 0x561a8ed78110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3003fff640 1 -- 172.21.15.92:0/480278059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed73c60 msgr2=0x561a8ed7f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3003fff640 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed73c60 0x561a8ed7f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3003fff640 1 -- 172.21.15.92:0/480278059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a8ec8dc60 con 0x561a8ed6eaf0 2024-03-19T02:01:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f3003fff640 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed78650 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ff002c170 tx=0x7f2ff002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3f9b6d20ee7368 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f2fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:01:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f2fe3fff640 1 -- 172.21.15.92:0/480278059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00053d0 con 0x561a8ed6eaf0 2024-03-19T02:01:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f2fe3fff640 1 -- 172.21.15.92:0/480278059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2ff0005570 con 0x561a8ed6eaf0 2024-03-19T02:01:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f300d9e0740 1 -- 172.21.15.92:0/480278059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff80041d0 con 0x561a8ed6eaf0 2024-03-19T02:01:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f2fe3fff640 1 -- 172.21.15.92:0/480278059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ff00046c0 con 0x561a8ed6eaf0 2024-03-19T02:01:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f300d9e0740 10 client.15978 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.854+0000 7f300d9e0740 1 -- 172.21.15.92:0/480278059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a8e9e7210 con 0x561a8ed6eaf0 2024-03-19T02:01:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f2fe3fff640 1 -- 172.21.15.92:0/480278059 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff80041d0 con 0x561a8ed6eaf0 2024-03-19T02:01:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f2fe3fff640 1 -- 172.21.15.92:0/480278059 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2ff0003b90 con 0x561a8ed6eaf0 2024-03-19T02:01:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f300d9e0740 10 client.15978 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f300d9e0740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f300d9e0740 20 client.15978 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f300d9e0740 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561a8ed84050 0x561a8ed86460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f300d9e0740 4 client.15978 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:01:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f300d9e0740 1 -- 172.21.15.92:0/480278059 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a8ece0120 con 0x561a8ed84050 2024-03-19T02:01:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f3008ab9640 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561a8ed84050 0x561a8ed86460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f2fe3fff640 10 client.15978 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.855+0000 7f3008ab9640 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561a8ed84050 0x561a8ed86460 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f39c95270b99de4b server_cookie=2f4338c80f7d7597 in_seq=0 out_seq=0 2024-03-19T02:01:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f2fe3fff640 1 -- 172.21.15.92:0/480278059 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561a8ece0120 con 0x561a8ed84050 2024-03-19T02:01:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f2fe3fff640 10 client.15978 handle_command_reply: tid=0 2024-03-19T02:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 client.15978 shutdown 2024-03-19T02:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 -- 172.21.15.92:0/480278059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 msgr2=0x561a8ed78650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed78650 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ff002c170 tx=0x7f2ff002bc30 comp rx=0 tx=0).stop 2024-03-19T02:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 -- 172.21.15.92:0/480278059 shutdown_connections 2024-03-19T02:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561a8ed84050 0x561a8ed86460 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed73c60 0x561a8ed7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed6eaf0 0x561a8ed78650 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f2fe3fff640 0 client.15978 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 --2- 172.21.15.92:0/480278059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 0x561a8ed78110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 -- 172.21.15.92:0/480278059 >> 172.21.15.92:0/480278059 conn(0x561a8ed69670 msgr2=0x561a8ed6a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 -- 172.21.15.92:0/480278059 shutdown_connections 2024-03-19T02:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 1 -- 172.21.15.92:0/480278059 wait complete. 2024-03-19T02:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.856+0000 7f300d9e0740 20 client.15978 trim_cache size 0 max 16384 2024-03-19T02:01:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f300d9e0740 1 Processor -- start 2024-03-19T02:01:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f300d9e0740 1 -- start start 2024-03-19T02:01:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 0x561a8ed68ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6eaf0 0x561a8ed7f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed6cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f300d9e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30041a8b00 con 0x561a8ed73c60 2024-03-19T02:01:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f300d9e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30041aa640 con 0x561a8ed6eaf0 2024-03-19T02:01:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f300d9e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30041a91d0 con 0x561a8ed6e100 2024-03-19T02:01:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f30092ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed6cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f3003fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6eaf0 0x561a8ed7f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f30092ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed6cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45046/0 (socket says 172.21.15.92:45046) 2024-03-19T02:01:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f30092ba640 1 -- 172.21.15.92:0/1483977751 learned_addr learned my addr 172.21.15.92:0/1483977751 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f3008ab9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 0x561a8ed68ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f30092ba640 1 -- 172.21.15.92:0/1483977751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 msgr2=0x561a8ed68ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f30092ba640 1 --2- 172.21.15.92:0/1483977751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 0x561a8ed68ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f30092ba640 1 -- 172.21.15.92:0/1483977751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6eaf0 msgr2=0x561a8ed7f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f30092ba640 1 --2- 172.21.15.92:0/1483977751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6eaf0 0x561a8ed7f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.877+0000 7f30092ba640 1 -- 172.21.15.92:0/1483977751 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8eab3620 con 0x561a8ed73c60 2024-03-19T02:01:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f30092ba640 1 --2- 172.21.15.92:0/1483977751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed6cfe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc09b5e0 tx=0x7f2ffc086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1516cd0e438608fa server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f2fee7fc640 1 -- 172.21.15.92:0/1483977751 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc09fa30 con 0x561a8ed73c60 2024-03-19T02:01:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f2fee7fc640 1 -- 172.21.15.92:0/1483977751 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc0a4080 con 0x561a8ed73c60 2024-03-19T02:01:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f2fee7fc640 1 -- 172.21.15.92:0/1483977751 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0a3bf0 con 0x561a8ed73c60 2024-03-19T02:01:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f300d9e0740 1 -- 172.21.15.92:0/1483977751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 msgr2=0x561a8ed6cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f300d9e0740 1 --2- 172.21.15.92:0/1483977751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed6cfe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc09b5e0 tx=0x7f2ffc086040 comp rx=0 tx=0).stop 2024-03-19T02:01:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f300d9e0740 1 -- 172.21.15.92:0/1483977751 shutdown_connections 2024-03-19T02:01:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f300d9e0740 1 --2- 172.21.15.92:0/1483977751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed6cfe0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f300d9e0740 1 --2- 172.21.15.92:0/1483977751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6eaf0 0x561a8ed7f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f300d9e0740 1 --2- 172.21.15.92:0/1483977751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6e100 0x561a8ed68ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f300d9e0740 1 -- 172.21.15.92:0/1483977751 >> 172.21.15.92:0/1483977751 conn(0x561a8ed69670 msgr2=0x561a8ed6c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f300d9e0740 1 -- 172.21.15.92:0/1483977751 shutdown_connections 2024-03-19T02:01:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.878+0000 7f300d9e0740 1 -- 172.21.15.92:0/1483977751 wait complete. 2024-03-19T02:01:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f300d9e0740 1 -- start start 2024-03-19T02:01:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f300d9e0740 1 Processor -- start 2024-03-19T02:01:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 0x561a8ed7d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6eaf0 0x561a8ed7d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f300d9e0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed7e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f300d9e0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30041a9f70 con 0x561a8ed73c60 2024-03-19T02:01:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f300d9e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30041a98a0 con 0x561a8ed6e100 2024-03-19T02:01:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f300d9e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30041a6ae0 con 0x561a8ed6eaf0 2024-03-19T02:01:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f300d9e0740 10 client.0 _ll_register_callbacks cb 0x7f2ff0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f30092ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed7e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f30092ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed7e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45050/0 (socket says 172.21.15.92:45050) 2024-03-19T02:01:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f30092ba640 1 -- 172.21.15.92:0/350909892 learned_addr learned my addr 172.21.15.92:0/350909892 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f3003fff640 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6eaf0 0x561a8ed7d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.879+0000 7f3008ab9640 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 0x561a8ed7d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f30092ba640 1 -- 172.21.15.92:0/350909892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6eaf0 msgr2=0x561a8ed7d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f30092ba640 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6eaf0 0x561a8ed7d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f30092ba640 1 -- 172.21.15.92:0/350909892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 msgr2=0x561a8ed7d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f30092ba640 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 0x561a8ed7d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f30092ba640 1 -- 172.21.15.92:0/350909892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a8ed7eb40 con 0x561a8ed73c60 2024-03-19T02:01:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f30092ba640 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed7e550 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0ce010 tx=0x7f2ffc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6717afacec46a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f2fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:01:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f2fe3fff640 1 -- 172.21.15.92:0/350909892 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc07e020 con 0x561a8ed73c60 2024-03-19T02:01:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f2fe3fff640 1 -- 172.21.15.92:0/350909892 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2ffc0a3bf0 con 0x561a8ed73c60 2024-03-19T02:01:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f300d9e0740 1 -- 172.21.15.92:0/350909892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ffc0a4080 con 0x561a8ed73c60 2024-03-19T02:01:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f2fe3fff640 1 -- 172.21.15.92:0/350909892 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ffc0a3e30 con 0x561a8ed73c60 2024-03-19T02:01:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f300d9e0740 10 client.15984 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f300d9e0740 1 -- 172.21.15.92:0/350909892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a8edea660 con 0x561a8ed73c60 2024-03-19T02:01:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f2fe3fff640 1 -- 172.21.15.92:0/350909892 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc0a4080 con 0x561a8ed73c60 2024-03-19T02:01:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f2fe3fff640 1 -- 172.21.15.92:0/350909892 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2ffc09a7a0 con 0x561a8ed73c60 2024-03-19T02:01:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f300d9e0740 10 client.15984 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f300d9e0740 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f300d9e0740 20 client.15984 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f300d9e0740 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561a8edeb0b0 0x561a8eded4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f300d9e0740 4 client.15984 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.880+0000 7f300d9e0740 1 -- 172.21.15.92:0/350909892 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a8ecda470 con 0x561a8edeb0b0 2024-03-19T02:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.881+0000 7f3008ab9640 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561a8edeb0b0 0x561a8eded4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.881+0000 7f3008ab9640 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561a8edeb0b0 0x561a8eded4d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914af0d7d8ea8704 server_cookie=15876ae504d86cee in_seq=0 out_seq=0 2024-03-19T02:01:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.881+0000 7f2fe3fff640 10 client.15984 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f2fe3fff640 1 -- 172.21.15.92:0/350909892 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a8ecda470 con 0x561a8edeb0b0 2024-03-19T02:01:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f2fe3fff640 10 client.15984 handle_command_reply: tid=0 2024-03-19T02:01:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 client.15984 shutdown 2024-03-19T02:01:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 -- 172.21.15.92:0/350909892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 msgr2=0x561a8ed7e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed7e550 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0ce010 tx=0x7f2ffc00c970 comp rx=0 tx=0).stop 2024-03-19T02:01:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 -- 172.21.15.92:0/350909892 shutdown_connections 2024-03-19T02:01:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x561a8edeb0b0 0x561a8eded4d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561a8ed73c60 0x561a8ed7e550 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a8ed6eaf0 0x561a8ed7d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f2fe3fff640 0 client.15984 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 --2- 172.21.15.92:0/350909892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a8ed6e100 0x561a8ed7d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 -- 172.21.15.92:0/350909892 >> 172.21.15.92:0/350909892 conn(0x561a8ed69670 msgr2=0x561a8ed6d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 -- 172.21.15.92:0/350909892 shutdown_connections 2024-03-19T02:01:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 1 -- 172.21.15.92:0/350909892 wait complete. 2024-03-19T02:01:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.887+0000 7f300d9e0740 20 client.15984 trim_cache size 0 max 16384 2024-03-19T02:01:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:01:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:01:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:01:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 -- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ff40a2920 msgr2=0x7f2ff40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ff40a2920 0x7f2ff40a4de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0036040 tx=0x7f2ff0005d50 comp rx=0 tx=0).stop 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 -- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30041022f0 msgr2=0x7f300419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30041022f0 0x7f300419ee40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00eb10 tx=0x7f2ffc00c2d0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 -- 172.21.15.92:0/2343442073 shutdown_connections 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30041022f0 0x7f300419ee40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ff40a2920 0x7f2ff40a4de0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3004101930 0x7f300419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 --2- 172.21.15.92:0/2343442073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3004100740 0x7f3004069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 -- 172.21.15.92:0/2343442073 >> 172.21.15.92:0/2343442073 conn(0x7f30040fbcb0 msgr2=0x7f3004104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 -- 172.21.15.92:0/2343442073 shutdown_connections 2024-03-19T02:01:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:00.888+0000 7f300ad1d640 1 -- 172.21.15.92:0/2343442073 wait complete. 2024-03-19T02:01:00.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:00 smithi159 ceph-mon[67110]: pgmap v597: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:01:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:00 smithi092 ceph-mon[62574]: pgmap v597: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:01:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:00 smithi144 ceph-mon[67199]: pgmap v597: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:01:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:02 smithi159 ceph-mon[67110]: pgmap v598: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T02:01:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:02 smithi092 ceph-mon[62574]: pgmap v598: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T02:01:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:02 smithi144 ceph-mon[67199]: pgmap v598: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T02:01:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:04 smithi159 ceph-mon[67110]: pgmap v599: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T02:01:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:04 smithi092 ceph-mon[62574]: pgmap v599: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T02:01:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:04 smithi144 ceph-mon[67199]: pgmap v599: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-03-19T02:01:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:06 smithi159 ceph-mon[67110]: pgmap v600: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:01:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:06 smithi092 ceph-mon[62574]: pgmap v600: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:01:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:06 smithi144 ceph-mon[67199]: pgmap v600: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:01:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:08 smithi159 ceph-mon[67110]: pgmap v601: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:01:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:08 smithi092 ceph-mon[62574]: pgmap v601: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:01:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:08 smithi144 ceph-mon[67199]: pgmap v601: 97 pgs: 97 active+clean; 2.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:01:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:10 smithi159 ceph-mon[67110]: pgmap v602: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:10 smithi092 ceph-mon[62574]: pgmap v602: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:10 smithi144 ceph-mon[67199]: pgmap v602: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:12 smithi159 ceph-mon[67110]: pgmap v603: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:01:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:12 smithi092 ceph-mon[62574]: pgmap v603: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:01:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:12 smithi144 ceph-mon[67199]: pgmap v603: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:01:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 -- 172.21.15.92:0/3498254404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0103a80 msgr2=0x7f8fb0103ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/3498254404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0103a80 0x7f8fb0103ea0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009be0 tx=0x7f8fa802bef0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 -- 172.21.15.92:0/3498254404 shutdown_connections 2024-03-19T02:01:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/3498254404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fb0105630 0x7f8fb010bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/3498254404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0104c70 0x7f8fb01050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/3498254404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0103a80 0x7f8fb0103ea0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 -- 172.21.15.92:0/3498254404 >> 172.21.15.92:0/3498254404 conn(0x7f8fb00ff010 msgr2=0x7f8fb0101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 -- 172.21.15.92:0/3498254404 shutdown_connections 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 -- 172.21.15.92:0/3498254404 wait complete. 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 Processor -- start 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 -- start start 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.975+0000 7f8fb6e68640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fb0103a80 0x7f8fb0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fb6e68640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0104c70 0x7f8fb019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fb6e68640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0105630 0x7f8fb01a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fb6e68640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010f700 con 0x7f8fb0104c70 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fb6e68640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010f580 con 0x7f8fb0105630 2024-03-19T02:01:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fb6e68640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010f880 con 0x7f8fb0103a80 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8faffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0104c70 0x7f8fb019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fb4c04640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fb0103a80 0x7f8fb0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8faffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0104c70 0x7f8fb019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51542/0 (socket says 172.21.15.92:51542) 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8faffff640 1 -- 172.21.15.92:0/1691783262 learned_addr learned my addr 172.21.15.92:0/1691783262 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fb5405640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0105630 0x7f8fb01a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8faffff640 1 -- 172.21.15.92:0/1691783262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fb0103a80 msgr2=0x7f8fb0199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8faffff640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fb0103a80 0x7f8fb0199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8faffff640 1 -- 172.21.15.92:0/1691783262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0105630 msgr2=0x7f8fb01a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8faffff640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0105630 0x7f8fb01a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8faffff640 1 -- 172.21.15.92:0/1691783262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb01a1900 con 0x7f8fb0104c70 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8faffff640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0104c70 0x7f8fb019a200 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9f0 tx=0x7f8fa000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fadffb640 1 -- 172.21.15.92:0/1691783262 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000ccc0 con 0x7f8fb0104c70 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fb6e68640 1 -- 172.21.15.92:0/1691783262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb01a1bf0 con 0x7f8fb0104c70 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fadffb640 1 -- 172.21.15.92:0/1691783262 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa0010910 con 0x7f8fb0104c70 2024-03-19T02:01:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.976+0000 7f8fadffb640 1 -- 172.21.15.92:0/1691783262 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00188a0 con 0x7f8fb0104c70 2024-03-19T02:01:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.977+0000 7f8fb6e68640 1 -- 172.21.15.92:0/1691783262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb01a2190 con 0x7f8fb0104c70 2024-03-19T02:01:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fadffb640 1 -- 172.21.15.92:0/1691783262 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f8fa0010ac0 con 0x7f8fb0104c70 2024-03-19T02:01:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb9a4e740 1 Processor -- start 2024-03-19T02:01:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb9a4e740 1 -- start start 2024-03-19T02:01:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd52520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52af0 0x55c51fd575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 0x55c51fd5a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c51fcb6be0 con 0x55c51fd52100 2024-03-19T02:01:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c51fc5f960 con 0x55c51fd57c60 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c51fcb2c80 con 0x55c51fd52af0 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fadffb640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8f9c0a28a0 0x7f8f9c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb4c04640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52af0 0x55c51fd575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb4c04640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52af0 0x55c51fd575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:44132/0 (socket says 172.21.15.92:44132) 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb4c04640 1 -- 172.21.15.92:0/955405319 learned_addr learned my addr 172.21.15.92:0/955405319 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.979+0000 7f8fb5405640 1 --2- 172.21.15.92:0/955405319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd52520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8faffff640 1 --2- 172.21.15.92:0/955405319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 0x55c51fd5a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fb5405640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8f9c0a28a0 0x7f8f9c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fadffb640 1 -- 172.21.15.92:0/1691783262 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8fa00cee90 con 0x7f8fb0104c70 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fb4c04640 1 -- 172.21.15.92:0/955405319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 msgr2=0x55c51fd5a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/955405319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 0x55c51fd5a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fb4c04640 1 -- 172.21.15.92:0/955405319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 msgr2=0x55c51fd52520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/955405319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd52520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fb4c04640 1 -- 172.21.15.92:0/955405319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51fc56d20 con 0x55c51fd52af0 2024-03-19T02:01:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fb5405640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8f9c0a28a0 0x7f8f9c0a4d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8f980058c0 tx=0x7f8f98009840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/955405319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52af0 0x55c51fd575f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009be0 tx=0x7f8fa8005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41320452f0acc0d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fa6ffd640 1 -- 172.21.15.92:0/955405319 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8043030 con 0x55c51fd52af0 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fa6ffd640 1 -- 172.21.15.92:0/955405319 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa8033b90 con 0x55c51fd52af0 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.980+0000 7f8fa6ffd640 1 -- 172.21.15.92:0/955405319 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa802c890 con 0x55c51fd52af0 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/955405319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52af0 msgr2=0x55c51fd575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/955405319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52af0 0x55c51fd575f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009be0 tx=0x7f8fa8005eb0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/955405319 shutdown_connections 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/955405319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 0x55c51fd5a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/955405319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52af0 0x55c51fd575f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/955405319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd52520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/955405319 >> 172.21.15.92:0/955405319 conn(0x55c51fd4d670 msgr2=0x55c51fd4da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/955405319 shutdown_connections 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/955405319 wait complete. 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 -- start start 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.981+0000 7f8fb9a4e740 1 Processor -- start 2024-03-19T02:01:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 0x55c51fd5c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd5c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd57c60 0x55c51fd635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c51fd5abb0 con 0x55c51fd57c60 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c51fc72db0 con 0x55c51fd52af0 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c51fd5ad30 con 0x55c51fd52100 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb9a4e740 10 client.0 _ll_register_callbacks cb 0x55c51fc77e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8faffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd5c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb5405640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd57c60 0x55c51fd635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8faffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd5c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:44832/0 (socket says 172.21.15.92:44832) 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb5405640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd57c60 0x55c51fd635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51566/0 (socket says 172.21.15.92:51566) 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8faffff640 1 -- 172.21.15.92:0/668659827 learned_addr learned my addr 172.21.15.92:0/668659827 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 0x55c51fd5c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8faffff640 1 -- 172.21.15.92:0/668659827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 msgr2=0x55c51fd5c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8faffff640 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 0x55c51fd5c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8faffff640 1 -- 172.21.15.92:0/668659827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd57c60 msgr2=0x55c51fd635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8faffff640 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd57c60 0x55c51fd635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8faffff640 1 -- 172.21.15.92:0/668659827 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51fc71c60 con 0x55c51fd52af0 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8fb5405640 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd57c60 0x55c51fd635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8faffff640 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd5c590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0097ca0 tx=0x7f8fa006a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3fda80c5454c586 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8f8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.982+0000 7f8f8ffff640 1 -- 172.21.15.92:0/668659827 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00cce60 con 0x55c51fd52af0 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8f8ffff640 1 -- 172.21.15.92:0/668659827 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8fa006d020 con 0x55c51fd52af0 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/668659827 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa8033b90 con 0x55c51fd52af0 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8f8ffff640 1 -- 172.21.15.92:0/668659827 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8fa009bf40 con 0x55c51fd52af0 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb9a4e740 10 client.25829 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/668659827 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c51f9cb210 con 0x55c51fd52af0 2024-03-19T02:01:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8f8ffff640 1 -- 172.21.15.92:0/668659827 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa8033b90 con 0x55c51fd52af0 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8f8ffff640 1 -- 172.21.15.92:0/668659827 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8fa009ad20 con 0x55c51fd52af0 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb9a4e740 10 client.25829 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb9a4e740 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb9a4e740 20 client.25829 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c51fd67ec0 0x55c51fd6a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb9a4e740 4 client.25829 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/668659827 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c51fcc4120 con 0x55c51fd67ec0 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.983+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c51fd67ec0 0x55c51fd6a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.984+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c51fd67ec0 0x55c51fd6a2b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d04c5bc74914f77 server_cookie=c5eec35ce50029c1 in_seq=0 out_seq=0 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.984+0000 7f8f8ffff640 10 client.25829 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8f8ffff640 1 -- 172.21.15.92:0/668659827 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c51fcc4120 con 0x55c51fd67ec0 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8f8ffff640 10 client.25829 handle_command_reply: tid=0 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 client.25829 shutdown 2024-03-19T02:01:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/668659827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 msgr2=0x55c51fd5c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd5c590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0097ca0 tx=0x7f8fa006a6a0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/668659827 shutdown_connections 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c51fd67ec0 0x55c51fd6a2b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd57c60 0x55c51fd635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd5c590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8f8ffff640 0 client.25829 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/668659827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 0x55c51fd5c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/668659827 >> 172.21.15.92:0/668659827 conn(0x55c51fd4d670 msgr2=0x55c51fd4e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/668659827 shutdown_connections 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/668659827 wait complete. 2024-03-19T02:01:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.985+0000 7f8fb9a4e740 20 client.25829 trim_cache size 0 max 16384 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:13.999+0000 7f8fb9a4e740 1 Processor -- start 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb9a4e740 1 -- start start 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd4cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd63590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd57c60 0x55c51fd570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01aadb0 con 0x55c51fd52100 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01ac8f0 con 0x55c51fd52af0 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01ab480 con 0x55c51fd57c60 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb4c04640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd4cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb4c04640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd4cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51574/0 (socket says 172.21.15.92:51574) 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb4c04640 1 -- 172.21.15.92:0/421639069 learned_addr learned my addr 172.21.15.92:0/421639069 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb5405640 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd57c60 0x55c51fd570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8faffff640 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd63590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb4c04640 1 -- 172.21.15.92:0/421639069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd57c60 msgr2=0x55c51fd570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd57c60 0x55c51fd570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb4c04640 1 -- 172.21.15.92:0/421639069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 msgr2=0x55c51fd63590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd63590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb4c04640 1 -- 172.21.15.92:0/421639069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51fa97620 con 0x55c51fd52100 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8faffff640 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd63590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd4cab0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55c51fd53560 tx=0x7f8fa8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626a4f765966074d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fa67fc640 1 -- 172.21.15.92:0/421639069 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8043040 con 0x55c51fd52100 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fa67fc640 1 -- 172.21.15.92:0/421639069 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa8033b90 con 0x55c51fd52100 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.000+0000 7f8fa67fc640 1 -- 172.21.15.92:0/421639069 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8034690 con 0x55c51fd52100 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/421639069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 msgr2=0x55c51fd4cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd4cab0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55c51fd53560 tx=0x7f8fa8000f40 comp rx=0 tx=0).stop 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/421639069 shutdown_connections 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd57c60 0x55c51fd570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd52af0 0x55c51fd63590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/421639069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52100 0x55c51fd4cab0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/421639069 >> 172.21.15.92:0/421639069 conn(0x55c51fd4d670 msgr2=0x55c51fd53590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/421639069 shutdown_connections 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/421639069 wait complete. 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.001+0000 7f8fb9a4e740 1 -- start start 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb9a4e740 1 Processor -- start 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 0x55c51fd61460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52af0 0x55c51fd619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb9a4e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 0x55c51fd629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01ac220 con 0x55c51fd52af0 2024-03-19T02:01:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01abb50 con 0x55c51fd57c60 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb9a4e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01a8d90 con 0x55c51fd52100 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb9a4e740 10 client.0 _ll_register_callbacks cb 0x7f8fa000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8faffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52af0 0x55c51fd619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8faffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52af0 0x55c51fd619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51580/0 (socket says 172.21.15.92:51580) 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8faffff640 1 -- 172.21.15.92:0/670729609 learned_addr learned my addr 172.21.15.92:0/670729609 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 0x55c51fd61460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8fb5405640 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 0x55c51fd629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8faffff640 1 -- 172.21.15.92:0/670729609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 msgr2=0x55c51fd61460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8faffff640 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 0x55c51fd61460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8faffff640 1 -- 172.21.15.92:0/670729609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 msgr2=0x55c51fd629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8faffff640 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 0x55c51fd629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8faffff640 1 -- 172.21.15.92:0/670729609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51fd62fa0 con 0x55c51fd52af0 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8faffff640 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52af0 0x55c51fd619a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0099af0 tx=0x7f8fa009aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd232fb9ad9bcf8f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.002+0000 7f8f8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8f8ffff640 1 -- 172.21.15.92:0/670729609 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00cce60 con 0x55c51fd52af0 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8f8ffff640 1 -- 172.21.15.92:0/670729609 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8fa006e0d0 con 0x55c51fd52af0 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/670729609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa8033b90 con 0x55c51fd52af0 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8f8ffff640 1 -- 172.21.15.92:0/670729609 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8fa009b380 con 0x55c51fd52af0 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb9a4e740 10 client.16002 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/670729609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c51fdce8f0 con 0x55c51fd52af0 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8f8ffff640 1 -- 172.21.15.92:0/670729609 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa8033b90 con 0x55c51fd52af0 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8f8ffff640 1 -- 172.21.15.92:0/670729609 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8fa006a1a0 con 0x55c51fd52af0 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb9a4e740 10 client.16002 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb9a4e740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb9a4e740 20 client.16002 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c51fdcf2c0 0x55c51fdd16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb9a4e740 4 client.16002 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:01:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/670729609 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c51fcbe470 con 0x55c51fdcf2c0 2024-03-19T02:01:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.003+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c51fdcf2c0 0x55c51fdd16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.004+0000 7f8f8ffff640 10 client.16002 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.004+0000 7f8fb4c04640 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c51fdcf2c0 0x55c51fdd16e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e01d4e0bcb31ea server_cookie=260722f9a4f672ed in_seq=0 out_seq=0 2024-03-19T02:01:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8f8ffff640 1 -- 172.21.15.92:0/670729609 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c51fcbe470 con 0x55c51fdcf2c0 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8f8ffff640 10 client.16002 handle_command_reply: tid=0 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 client.16002 shutdown 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/670729609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52af0 msgr2=0x55c51fd619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52af0 0x55c51fd619a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0099af0 tx=0x7f8fa009aec0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/670729609 shutdown_connections 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c51fdcf2c0 0x55c51fdd16e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c51fd57c60 0x55c51fd629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c51fd52af0 0x55c51fd619a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8f8ffff640 0 client.16002 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 --2- 172.21.15.92:0/670729609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c51fd52100 0x55c51fd61460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/670729609 >> 172.21.15.92:0/670729609 conn(0x55c51fd4d670 msgr2=0x55c51fd53590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/670729609 shutdown_connections 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 1 -- 172.21.15.92:0/670729609 wait complete. 2024-03-19T02:01:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.010+0000 7f8fb9a4e740 20 client.16002 trim_cache size 0 max 16384 2024-03-19T02:01:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:01:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:01:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:01:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 -- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8f9c0a28a0 msgr2=0x7f8f9c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8f9c0a28a0 0x7f8f9c0a4d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8f980058c0 tx=0x7f8f98009840 comp rx=0 tx=0).stop 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 -- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0104c70 msgr2=0x7f8fb019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0104c70 0x7f8fb019a200 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9f0 tx=0x7f8fa000edf0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 -- 172.21.15.92:0/1691783262 shutdown_connections 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fb0105630 0x7f8fb01a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fb0104c70 0x7f8fb019a200 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f8f9c0a28a0 0x7f8f9c0a4d60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 --2- 172.21.15.92:0/1691783262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fb0103a80 0x7f8fb0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 -- 172.21.15.92:0/1691783262 >> 172.21.15.92:0/1691783262 conn(0x7f8fb00ff010 msgr2=0x7f8fb0107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 -- 172.21.15.92:0/1691783262 shutdown_connections 2024-03-19T02:01:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:14.011+0000 7f8fb6e68640 1 -- 172.21.15.92:0/1691783262 wait complete. 2024-03-19T02:01:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:14 smithi159 ceph-mon[67110]: pgmap v604: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:14 smithi092 ceph-mon[62574]: pgmap v604: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:14 smithi144 ceph-mon[67199]: pgmap v604: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:16 smithi159 ceph-mon[67110]: pgmap v605: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-19T02:01:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:16 smithi092 ceph-mon[62574]: pgmap v605: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-19T02:01:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:16 smithi144 ceph-mon[67199]: pgmap v605: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-19T02:01:18.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:18 smithi159 ceph-mon[67110]: pgmap v606: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-03-19T02:01:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:18 smithi092 ceph-mon[62574]: pgmap v606: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-03-19T02:01:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:18 smithi144 ceph-mon[67199]: pgmap v606: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-03-19T02:01:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:01:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:01:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:01:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:01:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:01:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:01:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:01:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:01:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:01:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:01:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:01:20.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:01:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:20 smithi092 ceph-mon[62574]: pgmap v607: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:01:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:20 smithi144 ceph-mon[67199]: pgmap v607: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:01:21.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:20 smithi159 ceph-mon[67110]: pgmap v607: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:01:23.007 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:22 smithi159 ceph-mon[67110]: pgmap v608: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:22 smithi092 ceph-mon[62574]: pgmap v608: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:22 smithi144 ceph-mon[67199]: pgmap v608: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:25.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:24 smithi092 ceph-mon[62574]: pgmap v609: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:25.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:24 smithi144 ceph-mon[67199]: pgmap v609: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:25.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:24 smithi159 ceph-mon[67110]: pgmap v609: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:26 smithi092 ceph-mon[62574]: pgmap v610: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:26 smithi144 ceph-mon[67199]: pgmap v610: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.098+0000 7f0028634640 1 -- 172.21.15.92:0/3689745802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 msgr2=0x7f0020108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.098+0000 7f0028634640 1 --2- 172.21.15.92:0/3689745802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 0x7f0020108b80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0018009ca0 tx=0x7f001802c0f0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 -- 172.21.15.92:0/3689745802 shutdown_connections 2024-03-19T02:01:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 --2- 172.21.15.92:0/3689745802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 0x7f0020108b80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 --2- 172.21.15.92:0/3689745802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0020101930 0x7f0020101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 --2- 172.21.15.92:0/3689745802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020100740 0x7f0020100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 -- 172.21.15.92:0/3689745802 >> 172.21.15.92:0/3689745802 conn(0x7f00200fbcb0 msgr2=0x7f00200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 -- 172.21.15.92:0/3689745802 shutdown_connections 2024-03-19T02:01:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 -- 172.21.15.92:0/3689745802 wait complete. 2024-03-19T02:01:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 Processor -- start 2024-03-19T02:01:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 -- start start 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020100740 0x7f0020196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0020101930 0x7f0020196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 0x7f002019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f002010c3c0 con 0x7f00201022f0 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f002010c240 con 0x7f0020101930 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0028634640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f002010c540 con 0x7f0020100740 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0026bd1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 0x7f002019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f00263d0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020100740 0x7f0020196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.099+0000 7f0026bd1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 0x7f002019def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58152/0 (socket says 172.21.15.92:58152) 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0026bd1640 1 -- 172.21.15.92:0/962708597 learned_addr learned my addr 172.21.15.92:0/962708597 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0025bcf640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0020101930 0x7f0020196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0026bd1640 1 -- 172.21.15.92:0/962708597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020100740 msgr2=0x7f0020196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0026bd1640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020100740 0x7f0020196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0026bd1640 1 -- 172.21.15.92:0/962708597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0020101930 msgr2=0x7f0020196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0026bd1640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0020101930 0x7f0020196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0026bd1640 1 -- 172.21.15.92:0/962708597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002019e560 con 0x7f00201022f0 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0026bd1640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 0x7f002019def0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0018037040 tx=0x7f0018032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f00177fe640 1 -- 172.21.15.92:0/962708597 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018041020 con 0x7f00201022f0 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f00177fe640 1 -- 172.21.15.92:0/962708597 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0018034cd0 con 0x7f00201022f0 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f00177fe640 1 -- 172.21.15.92:0/962708597 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018039040 con 0x7f00201022f0 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0028634640 1 -- 172.21.15.92:0/962708597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002019e7f0 con 0x7f00201022f0 2024-03-19T02:01:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.100+0000 7f0028634640 1 -- 172.21.15.92:0/962708597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002019ed30 con 0x7f00201022f0 2024-03-19T02:01:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.102+0000 7f00177fe640 1 -- 172.21.15.92:0/962708597 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f0018044020 con 0x7f00201022f0 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f002b21c740 1 Processor -- start 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f002b21c740 1 -- start start 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f002b21c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 0x56082b240730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f002b21c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b245800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f002b21c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b248300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f002b21c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56082b1a2be0 con 0x56082b245e70 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f002b21c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56082b14b960 con 0x56082b240d00 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f002b21c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56082b19ec80 con 0x56082b240310 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f0025bcf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b248300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f0025bcf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b248300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58162/0 (socket says 172.21.15.92:58162) 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f0025bcf640 1 -- 172.21.15.92:0/1009317445 learned_addr learned my addr 172.21.15.92:0/1009317445 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f00263d0640 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b245800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f0025bcf640 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 0x56082b240730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f00177fe640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f000c0a2920 0x7f000c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.103+0000 7f0026bd1640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f000c0a2920 0x7f000c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f00263d0640 1 -- 172.21.15.92:0/1009317445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 msgr2=0x56082b240730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f00263d0640 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 0x56082b240730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f00263d0640 1 -- 172.21.15.92:0/1009317445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 msgr2=0x56082b248300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f00263d0640 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b248300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f00263d0640 1 -- 172.21.15.92:0/1009317445 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082b142d20 con 0x56082b240d00 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f0025bcf640 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b248300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f00177fe640 1 -- 172.21.15.92:0/962708597 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f00180f0840 con 0x7f00201022f0 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f00263d0640 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b245800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f001c009bd0 tx=0x7f001c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38fb13e1f3c12313 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f0026bd1640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f000c0a2920 0x7f000c0a4de0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f00180bd210 tx=0x7f00180dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f0014bfe640 1 -- 172.21.15.92:0/1009317445 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c004230 con 0x56082b240d00 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f0014bfe640 1 -- 172.21.15.92:0/1009317445 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f001c009e40 con 0x56082b240d00 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.104+0000 7f0014bfe640 1 -- 172.21.15.92:0/1009317445 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c019750 con 0x56082b240d00 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 -- 172.21.15.92:0/1009317445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 msgr2=0x56082b245800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b245800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f001c009bd0 tx=0x7f001c011810 comp rx=0 tx=0).stop 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 -- 172.21.15.92:0/1009317445 shutdown_connections 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b248300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b245800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 --2- 172.21.15.92:0/1009317445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 0x56082b240730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 -- 172.21.15.92:0/1009317445 >> 172.21.15.92:0/1009317445 conn(0x56082b23b880 msgr2=0x56082b23bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 -- 172.21.15.92:0/1009317445 shutdown_connections 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 -- 172.21.15.92:0/1009317445 wait complete. 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 -- start start 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.105+0000 7f002b21c740 1 Processor -- start 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f002b21c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f002b21c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240d00 0x56082b24a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f002b21c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b2518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f002b21c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56082b248e20 con 0x56082b245e70 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f002b21c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56082b15edb0 con 0x56082b240310 2024-03-19T02:01:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f002b21c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56082b248fa0 con 0x56082b240d00 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f002b21c740 10 client.0 _ll_register_callbacks cb 0x56082b163e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f00263d0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240d00 0x56082b24a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0025bcf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f00263d0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240d00 0x56082b24a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:42798/0 (socket says 172.21.15.92:42798) 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f00263d0640 1 -- 172.21.15.92:0/3342221176 learned_addr learned my addr 172.21.15.92:0/3342221176 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0025bcf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:60394/0 (socket says 172.21.15.92:60394) 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0025bcf640 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b2518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0025bcf640 1 -- 172.21.15.92:0/3342221176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240d00 msgr2=0x56082b24a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0025bcf640 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240d00 0x56082b24a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0025bcf640 1 -- 172.21.15.92:0/3342221176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 msgr2=0x56082b2518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0025bcf640 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b2518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0025bcf640 1 -- 172.21.15.92:0/3342221176 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082b15dc60 con 0x56082b240310 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f00263d0640 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240d00 0x56082b24a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0025bcf640 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24a340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x56082b241560 tx=0x7f0010004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733b2d189732e19 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-19T02:01:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0009ffb640 1 -- 172.21.15.92:0/3342221176 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001001a040 con 0x56082b240310 2024-03-19T02:01:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f002b21c740 1 -- 172.21.15.92:0/3342221176 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56082b1b0120 con 0x56082b240310 2024-03-19T02:01:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.106+0000 7f0009ffb640 1 -- 172.21.15.92:0/3342221176 <== mon.2 v2:172.21.15.144:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0010004050 con 0x56082b240310 2024-03-19T02:01:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f0009ffb640 1 -- 172.21.15.92:0/3342221176 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0010004250 con 0x56082b240310 2024-03-19T02:01:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f002b21c740 10 client.25847 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f002b21c740 1 -- 172.21.15.92:0/3342221176 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56082aeb7210 con 0x56082b240310 2024-03-19T02:01:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f0009ffb640 1 -- 172.21.15.92:0/3342221176 <== mon.2 v2:172.21.15.144:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56082b1b0120 con 0x56082b240310 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f0009ffb640 1 -- 172.21.15.92:0/3342221176 <== mon.2 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0010011b20 con 0x56082b240310 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f002b21c740 10 client.25847 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f002b21c740 10 client.25847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f002b21c740 20 client.25847 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f002b21c740 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56082b2560f0 0x56082b258560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f002b21c740 4 client.25847 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f002b21c740 1 -- 172.21.15.92:0/3342221176 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56082b1aa470 con 0x56082b2560f0 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.107+0000 7f00263d0640 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56082b2560f0 0x56082b258560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f00263d0640 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56082b2560f0 0x56082b258560 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae2d0d5f6709c56 server_cookie=555a601deb5080d2 in_seq=0 out_seq=0 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f0009ffb640 10 client.25847 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f0009ffb640 1 -- 172.21.15.92:0/3342221176 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56082b1aa470 con 0x56082b2560f0 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f0009ffb640 10 client.25847 handle_command_reply: tid=0 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 client.25847 shutdown 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 -- 172.21.15.92:0/3342221176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 msgr2=0x56082b24a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24a340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x56082b241560 tx=0x7f0010004730 comp rx=0 tx=0).stop 2024-03-19T02:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 -- 172.21.15.92:0/3342221176 shutdown_connections 2024-03-19T02:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56082b2560f0 0x56082b258560 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b2518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f0009ffb640 0 client.25847 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240d00 0x56082b24a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 --2- 172.21.15.92:0/3342221176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24a340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 -- 172.21.15.92:0/3342221176 >> 172.21.15.92:0/3342221176 conn(0x56082b23b880 msgr2=0x56082b23cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 -- 172.21.15.92:0/3342221176 shutdown_connections 2024-03-19T02:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.108+0000 7f002b21c740 1 -- 172.21.15.92:0/3342221176 wait complete. 2024-03-19T02:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.109+0000 7f002b21c740 20 client.25847 trim_cache size 0 max 16384 2024-03-19T02:01:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f002b21c740 1 Processor -- start 2024-03-19T02:01:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f002b21c740 1 -- start start 2024-03-19T02:01:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f002b21c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 0x56082b24a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f002b21c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b251880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f002b21c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b23f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f002b21c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f00201a6970 con 0x56082b245e70 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f002b21c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00201a84b0 con 0x56082b240d00 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f002b21c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00201a7040 con 0x56082b240310 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f0025bcf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b23f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f0025bcf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b23f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58178/0 (socket says 172.21.15.92:58178) 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f0025bcf640 1 -- 172.21.15.92:0/2109880424 learned_addr learned my addr 172.21.15.92:0/2109880424 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f00263d0640 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b251880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.123+0000 7f0025bcf640 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 0x56082b24a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f0025bcf640 1 -- 172.21.15.92:0/2109880424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 msgr2=0x56082b24a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f0025bcf640 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 0x56082b24a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f0025bcf640 1 -- 172.21.15.92:0/2109880424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 msgr2=0x56082b251880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f0025bcf640 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b251880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f0025bcf640 1 -- 172.21.15.92:0/2109880424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082af83620 con 0x56082b245e70 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f00263d0640 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b251880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f0025bcf640 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b23f1f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0010009fd0 tx=0x7f0010005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbadec96605ac86b server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f000bfff640 1 -- 172.21.15.92:0/2109880424 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010031070 con 0x56082b245e70 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f000bfff640 1 -- 172.21.15.92:0/2109880424 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0010004090 con 0x56082b245e70 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f000bfff640 1 -- 172.21.15.92:0/2109880424 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010005020 con 0x56082b245e70 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f002b21c740 1 -- 172.21.15.92:0/2109880424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 msgr2=0x56082b23f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.124+0000 7f002b21c740 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b23f1f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0010009fd0 tx=0x7f0010005f90 comp rx=0 tx=0).stop 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 -- 172.21.15.92:0/2109880424 shutdown_connections 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b245e70 0x56082b23f1f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240d00 0x56082b251880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 --2- 172.21.15.92:0/2109880424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b240310 0x56082b24a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 -- 172.21.15.92:0/2109880424 >> 172.21.15.92:0/2109880424 conn(0x56082b23b880 msgr2=0x56082b23eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 -- 172.21.15.92:0/2109880424 shutdown_connections 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 -- 172.21.15.92:0/2109880424 wait complete. 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 -- start start 2024-03-19T02:01:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 Processor -- start 2024-03-19T02:01:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b240d00 0x56082b24ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b245e70 0x56082b24fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f00201a7de0 con 0x56082b240d00 2024-03-19T02:01:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00201a7710 con 0x56082b240310 2024-03-19T02:01:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00201a4950 con 0x56082b245e70 2024-03-19T02:01:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f002b21c740 10 client.0 _ll_register_callbacks cb 0x7f0010009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f00263d0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b240d00 0x56082b24ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f00263d0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b240d00 0x56082b24ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58190/0 (socket says 172.21.15.92:58190) 2024-03-19T02:01:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f00263d0640 1 -- 172.21.15.92:0/1219540579 learned_addr learned my addr 172.21.15.92:0/1219540579 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f0025bcf640 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f0025bcf640 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b245e70 0x56082b24fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f00263d0640 1 -- 172.21.15.92:0/1219540579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b245e70 msgr2=0x56082b24fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f00263d0640 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b245e70 0x56082b24fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f00263d0640 1 -- 172.21.15.92:0/1219540579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 msgr2=0x56082b24e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f00263d0640 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f00263d0640 1 -- 172.21.15.92:0/1219540579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082b250480 con 0x56082b240d00 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.125+0000 7f0025bcf640 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b245e70 0x56082b24fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f00263d0640 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b240d00 0x56082b24ee80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f001c009bd0 tx=0x7f001c0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7bd1232dd2be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f0009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f0009ffb640 1 -- 172.21.15.92:0/1219540579 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c009800 con 0x56082b240d00 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f002b21c740 1 -- 172.21.15.92:0/1219540579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56082b128770 con 0x56082b240d00 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f0009ffb640 1 -- 172.21.15.92:0/1219540579 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f001c005350 con 0x56082b240d00 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f0009ffb640 1 -- 172.21.15.92:0/1219540579 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f001c005570 con 0x56082b240d00 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f002b21c740 10 client.16020 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f002b21c740 1 -- 172.21.15.92:0/1219540579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56082b251800 con 0x56082b240d00 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f0009ffb640 1 -- 172.21.15.92:0/1219540579 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56082b128770 con 0x56082b240d00 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f0009ffb640 1 -- 172.21.15.92:0/1219540579 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f001c003820 con 0x56082b240d00 2024-03-19T02:01:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f002b21c740 10 client.16020 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f002b21c740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f002b21c740 20 client.16020 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f002b21c740 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56082b2bc770 0x56082b2beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f002b21c740 4 client.16020 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:01:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.126+0000 7f002b21c740 1 -- 172.21.15.92:0/1219540579 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56082b174700 con 0x56082b2bc770 2024-03-19T02:01:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.127+0000 7f0025bcf640 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56082b2bc770 0x56082b2beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.127+0000 7f0009ffb640 10 client.16020 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.127+0000 7f0025bcf640 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56082b2bc770 0x56082b2beb60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359b38866ccbe4aa server_cookie=8634da43d6f9f737 in_seq=0 out_seq=0 2024-03-19T02:01:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f0009ffb640 1 -- 172.21.15.92:0/1219540579 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56082b174700 con 0x56082b2bc770 2024-03-19T02:01:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f0009ffb640 10 client.16020 handle_command_reply: tid=0 2024-03-19T02:01:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 client.16020 shutdown 2024-03-19T02:01:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 -- 172.21.15.92:0/1219540579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b240d00 msgr2=0x56082b24ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b240d00 0x56082b24ee80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f001c009bd0 tx=0x7f001c0044a0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 -- 172.21.15.92:0/1219540579 shutdown_connections 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x56082b2bc770 0x56082b2beb60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082b245e70 0x56082b24fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56082b240d00 0x56082b24ee80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 --2- 172.21.15.92:0/1219540579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56082b240310 0x56082b24e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 -- 172.21.15.92:0/1219540579 >> 172.21.15.92:0/1219540579 conn(0x56082b23b880 msgr2=0x56082b24a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f0009ffb640 0 client.16020 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 -- 172.21.15.92:0/1219540579 shutdown_connections 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 1 -- 172.21.15.92:0/1219540579 wait complete. 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.133+0000 7f002b21c740 20 client.16020 trim_cache size 0 max 16384 2024-03-19T02:01:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:01:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:01:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:01:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:01:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 -- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f000c0a2920 msgr2=0x7f000c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f000c0a2920 0x7f000c0a4de0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f00180bd210 tx=0x7f00180dd040 comp rx=0 tx=0).stop 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 -- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 msgr2=0x7f002019def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 0x7f002019def0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0018037040 tx=0x7f0018032f40 comp rx=0 tx=0).stop 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 -- 172.21.15.92:0/962708597 shutdown_connections 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00201022f0 0x7f002019def0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0020101930 0x7f0020196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f000c0a2920 0x7f000c0a4de0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 --2- 172.21.15.92:0/962708597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0020100740 0x7f0020196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 -- 172.21.15.92:0/962708597 >> 172.21.15.92:0/962708597 conn(0x7f00200fbcb0 msgr2=0x7f0020104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 -- 172.21.15.92:0/962708597 shutdown_connections 2024-03-19T02:01:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:27.134+0000 7f0028634640 1 -- 172.21.15.92:0/962708597 wait complete. 2024-03-19T02:01:27.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:26 smithi159 ceph-mon[67110]: pgmap v610: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:28 smithi092 ceph-mon[62574]: pgmap v611: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T02:01:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:28 smithi144 ceph-mon[67199]: pgmap v611: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T02:01:29.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:28 smithi159 ceph-mon[67110]: pgmap v611: 97 pgs: 97 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T02:01:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:30 smithi092 ceph-mon[62574]: pgmap v612: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T02:01:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:30 smithi144 ceph-mon[67199]: pgmap v612: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T02:01:31.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:30 smithi159 ceph-mon[67110]: pgmap v612: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T02:01:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:32 smithi092 ceph-mon[62574]: pgmap v613: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T02:01:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:32 smithi144 ceph-mon[67199]: pgmap v613: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T02:01:33.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:32 smithi159 ceph-mon[67110]: pgmap v613: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T02:01:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:34 smithi092 ceph-mon[62574]: pgmap v614: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:01:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:34 smithi144 ceph-mon[67199]: pgmap v614: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:01:35.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:34 smithi159 ceph-mon[67110]: pgmap v614: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:01:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:36 smithi092 ceph-mon[62574]: pgmap v615: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:01:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:36 smithi144 ceph-mon[67199]: pgmap v615: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:01:37.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:36 smithi159 ceph-mon[67110]: pgmap v615: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:01:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:38 smithi092 ceph-mon[62574]: pgmap v616: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:01:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:38 smithi144 ceph-mon[67199]: pgmap v616: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:01:39.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:38 smithi159 ceph-mon[67110]: pgmap v616: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:01:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 -- 172.21.15.92:0/713586749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd081033c0 msgr2=0x7efd08109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/713586749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd081033c0 0x7efd08109c50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efd00009ca0 tx=0x7efd0002c0f0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 -- 172.21.15.92:0/713586749 shutdown_connections 2024-03-19T02:01:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/713586749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd081033c0 0x7efd08109c50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/713586749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd08102a00 0x7efd08102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/713586749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd08101810 0x7efd08101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 -- 172.21.15.92:0/713586749 >> 172.21.15.92:0/713586749 conn(0x7efd080fcda0 msgr2=0x7efd080ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 -- 172.21.15.92:0/713586749 shutdown_connections 2024-03-19T02:01:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 -- 172.21.15.92:0/713586749 wait complete. 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 Processor -- start 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.222+0000 7efd0cbb9640 1 -- start start 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0cbb9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd08101810 0x7efd081979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0cbb9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd08102a00 0x7efd08197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0cbb9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd081033c0 0x7efd0819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0cbb9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810d470 con 0x7efd08102a00 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0cbb9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810d2f0 con 0x7efd08101810 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0cbb9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810d5f0 con 0x7efd081033c0 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd08102a00 0x7efd08197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd08102a00 0x7efd08197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50748/0 (socket says 172.21.15.92:50748) 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efcfffff640 1 -- 172.21.15.92:0/3418371784 learned_addr learned my addr 172.21.15.92:0/3418371784 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0659c640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd08101810 0x7efd081979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd06d9d640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd081033c0 0x7efd0819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0659c640 1 -- 172.21.15.92:0/3418371784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd081033c0 msgr2=0x7efd0819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0659c640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd081033c0 0x7efd0819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0659c640 1 -- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd08102a00 msgr2=0x7efd08197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0659c640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd08102a00 0x7efd08197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0659c640 1 -- 172.21.15.92:0/3418371784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0819f610 con 0x7efd08101810 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efcfffff640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd08102a00 0x7efd08197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd06d9d640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd081033c0 0x7efd0819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0659c640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd08101810 0x7efd081979f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e9c0 tx=0x7efcf800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efcff7fe640 1 -- 172.21.15.92:0/3418371784 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf800cd80 con 0x7efd08101810 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efd0cbb9640 1 -- 172.21.15.92:0/3418371784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0819f900 con 0x7efd08101810 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efcff7fe640 1 -- 172.21.15.92:0/3418371784 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf8010910 con 0x7efd08101810 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.223+0000 7efcff7fe640 1 -- 172.21.15.92:0/3418371784 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8018980 con 0x7efd08101810 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.224+0000 7efd0cbb9640 1 -- 172.21.15.92:0/3418371784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0819fea0 con 0x7efd08101810 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.226+0000 7efcff7fe640 1 -- 172.21.15.92:0/3418371784 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7efcf801f020 con 0x7efd08101810 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.226+0000 7efd0f7a4740 1 Processor -- start 2024-03-19T02:01:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd0f7a4740 1 -- start start 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 0x557aab97f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97faf0 0x557aab9845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab984c60 0x557aab9870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557aab8e3be0 con 0x557aab984c60 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557aab88c960 con 0x557aab97faf0 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557aab8dfc80 con 0x557aab97f100 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab984c60 0x557aab9870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab984c60 0x557aab9870f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50756/0 (socket says 172.21.15.92:50756) 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcfffff640 1 -- 172.21.15.92:0/442921664 learned_addr learned my addr 172.21.15.92:0/442921664 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd06d9d640 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 0x557aab97f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd0659c640 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97faf0 0x557aab9845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcff7fe640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efcf40a28a0 0x7efcf40a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcfffff640 1 -- 172.21.15.92:0/442921664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 msgr2=0x557aab97f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcfffff640 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 0x557aab97f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcfffff640 1 -- 172.21.15.92:0/442921664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97faf0 msgr2=0x557aab9845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcfffff640 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97faf0 0x557aab9845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcfffff640 1 -- 172.21.15.92:0/442921664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aab883d20 con 0x557aab984c60 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd06d9d640 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 0x557aab97f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efd06d9d640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efcf40a28a0 0x7efcf40a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.227+0000 7efcff7fe640 1 -- 172.21.15.92:0/3418371784 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efcf80cea40 con 0x7efd08101810 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efcfffff640 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab984c60 0x557aab9870f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efcf00115f0 tx=0x7efcf0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bf85de54e05676 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efcfcff9640 1 -- 172.21.15.92:0/442921664 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf001e070 con 0x557aab984c60 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efcfcff9640 1 -- 172.21.15.92:0/442921664 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf000f660 con 0x557aab984c60 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efcfcff9640 1 -- 172.21.15.92:0/442921664 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0003850 con 0x557aab984c60 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efd0f7a4740 1 -- 172.21.15.92:0/442921664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab984c60 msgr2=0x557aab9870f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab984c60 0x557aab9870f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efcf00115f0 tx=0x7efcf0011ab0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efd06d9d640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efcf40a28a0 0x7efcf40a4d60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efd00005e30 tx=0x7efd0002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efd0f7a4740 1 -- 172.21.15.92:0/442921664 shutdown_connections 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab984c60 0x557aab9870f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97faf0 0x557aab9845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/442921664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 0x557aab97f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.228+0000 7efd0f7a4740 1 -- 172.21.15.92:0/442921664 >> 172.21.15.92:0/442921664 conn(0x557aab97a670 msgr2=0x557aab97aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 -- 172.21.15.92:0/442921664 shutdown_connections 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 -- 172.21.15.92:0/442921664 wait complete. 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 -- start start 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 Processor -- start 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 0x557aab9890b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab984c60 0x557aab990630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557aab987c10 con 0x557aab97faf0 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557aab89fdb0 con 0x557aab984c60 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557aab987d90 con 0x557aab97f100 2024-03-19T02:01:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0f7a4740 10 client.0 _ll_register_callbacks cb 0x557aab8a4e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9895f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50772/0 (socket says 172.21.15.92:50772) 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efcfffff640 1 -- 172.21.15.92:0/2362765281 learned_addr learned my addr 172.21.15.92:0/2362765281 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd0659c640 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 0x557aab9890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.229+0000 7efd06d9d640 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab984c60 0x557aab990630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcfffff640 1 -- 172.21.15.92:0/2362765281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 msgr2=0x557aab9890b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcfffff640 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 0x557aab9890b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcfffff640 1 -- 172.21.15.92:0/2362765281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab984c60 msgr2=0x557aab990630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcfffff640 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab984c60 0x557aab990630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcfffff640 1 -- 172.21.15.92:0/2362765281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aab89ec60 con 0x557aab97faf0 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efd06d9d640 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab984c60 0x557aab990630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcfffff640 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9895f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efcf001c040 tx=0x7efcf000fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef6288e89eb6559 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcedffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcedffb640 1 -- 172.21.15.92:0/2362765281 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0026020 con 0x557aab97faf0 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcedffb640 1 -- 172.21.15.92:0/2362765281 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efcf00051c0 con 0x557aab97faf0 2024-03-19T02:01:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2362765281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf000f660 con 0x557aab97faf0 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcedffb640 1 -- 172.21.15.92:0/2362765281 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efcf001e040 con 0x557aab97faf0 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efd0f7a4740 10 client.16032 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2362765281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557aab5f8210 con 0x557aab97faf0 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcedffb640 1 -- 172.21.15.92:0/2362765281 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf000f660 con 0x557aab97faf0 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efcedffb640 1 -- 172.21.15.92:0/2362765281 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efcf0021030 con 0x557aab97faf0 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efd0f7a4740 10 client.16032 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.230+0000 7efd0f7a4740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.231+0000 7efd0f7a4740 20 client.16032 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.231+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557aab994d00 0x557aab9970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.231+0000 7efd0f7a4740 4 client.16032 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.231+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2362765281 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aab8f1120 con 0x557aab994d00 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.231+0000 7efd0659c640 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557aab994d00 0x557aab9970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.231+0000 7efcedffb640 10 client.16032 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.231+0000 7efd0659c640 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557aab994d00 0x557aab9970f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e404e20a693e669 server_cookie=139df2aefed8095c in_seq=0 out_seq=0 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efcedffb640 1 -- 172.21.15.92:0/2362765281 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557aab8f1120 con 0x557aab994d00 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efcedffb640 10 client.16032 handle_command_reply: tid=0 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 client.16032 shutdown 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2362765281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 msgr2=0x557aab9895f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9895f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efcf001c040 tx=0x7efcf000fef0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2362765281 shutdown_connections 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557aab994d00 0x557aab9970f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab984c60 0x557aab990630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9895f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2362765281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab97f100 0x557aab9890b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efcedffb640 0 client.16032 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2362765281 >> 172.21.15.92:0/2362765281 conn(0x557aab97a670 msgr2=0x557aab97b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2362765281 shutdown_connections 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2362765281 wait complete. 2024-03-19T02:01:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.232+0000 7efd0f7a4740 20 client.16032 trim_cache size 0 max 16384 2024-03-19T02:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.246+0000 7efd0f7a4740 1 Processor -- start 2024-03-19T02:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.246+0000 7efd0f7a4740 1 -- start start 2024-03-19T02:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 0x557aab9840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9905f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 0x557aab97dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd081a8bf0 con 0x557aab97faf0 2024-03-19T02:01:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd081aa730 con 0x557aab97f100 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd081a92c0 con 0x557aab984c60 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9905f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9905f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50786/0 (socket says 172.21.15.92:50786) 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efcfffff640 1 -- 172.21.15.92:0/1809832036 learned_addr learned my addr 172.21.15.92:0/1809832036 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efd06d9d640 1 --2- 172.21.15.92:0/1809832036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 0x557aab97dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efd0659c640 1 --2- 172.21.15.92:0/1809832036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 0x557aab9840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efcfffff640 1 -- 172.21.15.92:0/1809832036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 msgr2=0x557aab97dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efcfffff640 1 --2- 172.21.15.92:0/1809832036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 0x557aab97dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efcfffff640 1 -- 172.21.15.92:0/1809832036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 msgr2=0x557aab9840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efcfffff640 1 --2- 172.21.15.92:0/1809832036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 0x557aab9840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efcfffff640 1 -- 172.21.15.92:0/1809832036 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aab6c4620 con 0x557aab97faf0 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efcfffff640 1 --2- 172.21.15.92:0/1809832036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9905f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efcf002f040 tx=0x7efcf0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9aaae59e2b9721 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efceffff640 1 -- 172.21.15.92:0/1809832036 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0016070 con 0x557aab97faf0 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efceffff640 1 -- 172.21.15.92:0/1809832036 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf0005590 con 0x557aab97faf0 2024-03-19T02:01:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.247+0000 7efceffff640 1 -- 172.21.15.92:0/1809832036 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0026070 con 0x557aab97faf0 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 -- 172.21.15.92:0/1809832036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 msgr2=0x557aab9905f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/1809832036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9905f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efcf002f040 tx=0x7efcf0011ab0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 -- 172.21.15.92:0/1809832036 shutdown_connections 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/1809832036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 0x557aab97dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/1809832036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9905f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/1809832036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 0x557aab9840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 -- 172.21.15.92:0/1809832036 >> 172.21.15.92:0/1809832036 conn(0x557aab97a670 msgr2=0x557aab97d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 -- 172.21.15.92:0/1809832036 shutdown_connections 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 -- 172.21.15.92:0/1809832036 wait complete. 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 -- start start 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 Processor -- start 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 0x557aab98fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 0x557aab98e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd081aa060 con 0x557aab97faf0 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd081a9990 con 0x557aab97f100 2024-03-19T02:01:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd081a6bd0 con 0x557aab984c60 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.248+0000 7efd0f7a4740 10 client.0 _ll_register_callbacks cb 0x7efcf001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efd06d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 0x557aab98e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcfffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9903e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50802/0 (socket says 172.21.15.92:50802) 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcfffff640 1 -- 172.21.15.92:0/2891786766 learned_addr learned my addr 172.21.15.92:0/2891786766 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efd0659c640 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 0x557aab98fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcfffff640 1 -- 172.21.15.92:0/2891786766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 msgr2=0x557aab98e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcfffff640 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 0x557aab98e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcfffff640 1 -- 172.21.15.92:0/2891786766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 msgr2=0x557aab98fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcfffff640 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 0x557aab98fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcfffff640 1 -- 172.21.15.92:0/2891786766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aab98ec10 con 0x557aab97faf0 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcfffff640 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9903e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efcf0011510 tx=0x7efcf0005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9034b1c6263614b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcedffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcedffb640 1 -- 172.21.15.92:0/2891786766 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0016030 con 0x557aab97faf0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcedffb640 1 -- 172.21.15.92:0/2891786766 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efcf00052f0 con 0x557aab97faf0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2891786766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf0005590 con 0x557aab97faf0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efcedffb640 1 -- 172.21.15.92:0/2891786766 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efcf0026040 con 0x557aab97faf0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efd0f7a4740 10 client.16038 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.249+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2891786766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557aab9fba00 con 0x557aab97faf0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efcedffb640 1 -- 172.21.15.92:0/2891786766 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf0005590 con 0x557aab97faf0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efcedffb640 1 -- 172.21.15.92:0/2891786766 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efcf0034070 con 0x557aab97faf0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efd0f7a4740 10 client.16038 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efd0f7a4740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efd0f7a4740 20 client.16038 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557aab9fc240 0x557aab9fe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efd0f7a4740 4 client.16038 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2891786766 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557aab8eb470 con 0x557aab9fc240 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efd0659c640 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557aab9fc240 0x557aab9fe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efd0659c640 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557aab9fc240 0x557aab9fe660 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46c1f4d3b491f49 server_cookie=9a37bd1099dec3a6 in_seq=0 out_seq=0 2024-03-19T02:01:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.250+0000 7efcedffb640 10 client.16038 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efcedffb640 1 -- 172.21.15.92:0/2891786766 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557aab8eb470 con 0x557aab9fc240 2024-03-19T02:01:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efcedffb640 10 client.16038 handle_command_reply: tid=0 2024-03-19T02:01:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 client.16038 shutdown 2024-03-19T02:01:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2891786766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 msgr2=0x557aab9903e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9903e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efcf0011510 tx=0x7efcf0005970 comp rx=0 tx=0).stop 2024-03-19T02:01:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2891786766 shutdown_connections 2024-03-19T02:01:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x557aab9fc240 0x557aab9fe660 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557aab984c60 0x557aab98e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557aab97faf0 0x557aab9903e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efcedffb640 0 client.16038 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 --2- 172.21.15.92:0/2891786766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557aab97f100 0x557aab98fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2891786766 >> 172.21.15.92:0/2891786766 conn(0x557aab97a670 msgr2=0x557aab97d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2891786766 shutdown_connections 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 1 -- 172.21.15.92:0/2891786766 wait complete. 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.257+0000 7efd0f7a4740 20 client.16038 trim_cache size 0 max 16384 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 -- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efcf40a28a0 msgr2=0x7efcf40a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efcf40a28a0 0x7efcf40a4d60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efd00005e30 tx=0x7efd0002e040 comp rx=0 tx=0).stop 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 -- 172.21.15.92:0/3418371784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd08101810 msgr2=0x7efd081979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd08101810 0x7efd081979f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e9c0 tx=0x7efcf800ee80 comp rx=0 tx=0).stop 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 -- 172.21.15.92:0/3418371784 shutdown_connections 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7efcf40a28a0 0x7efcf40a4d60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd081033c0 0x7efd0819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd08102a00 0x7efd08197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 --2- 172.21.15.92:0/3418371784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd08101810 0x7efd081979f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 -- 172.21.15.92:0/3418371784 >> 172.21.15.92:0/3418371784 conn(0x7efd080fcda0 msgr2=0x7efd08105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 -- 172.21.15.92:0/3418371784 shutdown_connections 2024-03-19T02:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:40.258+0000 7efd0cbb9640 1 -- 172.21.15.92:0/3418371784 wait complete. 2024-03-19T02:01:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:40 smithi092 ceph-mon[62574]: pgmap v617: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:40 smithi144 ceph-mon[67199]: pgmap v617: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:41.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:40 smithi159 ceph-mon[67110]: pgmap v617: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:01:43.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:42 smithi092 ceph-mon[62574]: pgmap v618: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T02:01:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:42 smithi144 ceph-mon[67199]: pgmap v618: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T02:01:43.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:42 smithi159 ceph-mon[67110]: pgmap v618: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-03-19T02:01:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:44 smithi092 ceph-mon[62574]: pgmap v619: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T02:01:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:44 smithi144 ceph-mon[67199]: pgmap v619: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T02:01:45.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:44 smithi159 ceph-mon[67110]: pgmap v619: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-03-19T02:01:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:46 smithi092 ceph-mon[62574]: pgmap v620: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:46 smithi144 ceph-mon[67199]: pgmap v620: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:47.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:46 smithi159 ceph-mon[67110]: pgmap v620: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:48.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:47 smithi092 ceph-mon[62574]: pgmap v621: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:01:48.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:47 smithi144 ceph-mon[67199]: pgmap v621: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:01:48.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:47 smithi159 ceph-mon[67110]: pgmap v621: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:01:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:50 smithi159 ceph-mon[67110]: pgmap v622: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T02:01:51.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:50 smithi092 ceph-mon[62574]: pgmap v622: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T02:01:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:50 smithi144 ceph-mon[67199]: pgmap v622: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T02:01:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:52 smithi159 ceph-mon[67110]: pgmap v623: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:53.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:52 smithi092 ceph-mon[62574]: pgmap v623: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:52 smithi144 ceph-mon[67199]: pgmap v623: 97 pgs: 97 active+clean; 2.9 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.344+0000 7fcb69189640 1 -- 172.21.15.92:0/3978593204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb64100750 msgr2=0x7fcb64100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.344+0000 7fcb69189640 1 --2- 172.21.15.92:0/3978593204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb64100750 0x7fcb64100b70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009900 tx=0x7fcb5402bc30 comp rx=0 tx=0).stop 2024-03-19T02:01:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.344+0000 7fcb69189640 1 -- 172.21.15.92:0/3978593204 shutdown_connections 2024-03-19T02:01:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.344+0000 7fcb69189640 1 --2- 172.21.15.92:0/3978593204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb64102300 0x7fcb64108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.344+0000 7fcb69189640 1 --2- 172.21.15.92:0/3978593204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb64101940 0x7fcb64101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.344+0000 7fcb69189640 1 --2- 172.21.15.92:0/3978593204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb64100750 0x7fcb64100b70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.344+0000 7fcb69189640 1 -- 172.21.15.92:0/3978593204 >> 172.21.15.92:0/3978593204 conn(0x7fcb640fbcc0 msgr2=0x7fcb640fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.344+0000 7fcb69189640 1 -- 172.21.15.92:0/3978593204 shutdown_connections 2024-03-19T02:01:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.344+0000 7fcb69189640 1 -- 172.21.15.92:0/3978593204 wait complete. 2024-03-19T02:01:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb69189640 1 Processor -- start 2024-03-19T02:01:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb69189640 1 -- start start 2024-03-19T02:01:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb69189640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb64100750 0x7fcb6406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb69189640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb64101940 0x7fcb6419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb69189640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb64102300 0x7fcb6419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb69189640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410c3f0 con 0x7fcb64102300 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb69189640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410c270 con 0x7fcb64100750 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb69189640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6410c570 con 0x7fcb64101940 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb6259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb64101940 0x7fcb6419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb6359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb64102300 0x7fcb6419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb62d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb64100750 0x7fcb6406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb6259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb64101940 0x7fcb6419c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:37124/0 (socket says 172.21.15.92:37124) 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.345+0000 7fcb6259c640 1 -- 172.21.15.92:0/1030214518 learned_addr learned my addr 172.21.15.92:0/1030214518 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb62d9d640 1 -- 172.21.15.92:0/1030214518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb64101940 msgr2=0x7fcb6419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb64101940 0x7fcb6419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb62d9d640 1 -- 172.21.15.92:0/1030214518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb64102300 msgr2=0x7fcb6419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb64102300 0x7fcb6419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb62d9d640 1 -- 172.21.15.92:0/1030214518 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6419f760 con 0x7fcb64100750 2024-03-19T02:01:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb64100750 0x7fcb6406abd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009230 tx=0x7fcb5402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb5bfff640 1 -- 172.21.15.92:0/1030214518 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54043020 con 0x7fcb64100750 2024-03-19T02:01:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb69189640 1 -- 172.21.15.92:0/1030214518 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6419f9f0 con 0x7fcb64100750 2024-03-19T02:01:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb69189640 1 -- 172.21.15.92:0/1030214518 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6419ff60 con 0x7fcb64100750 2024-03-19T02:01:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb5bfff640 1 -- 172.21.15.92:0/1030214518 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb540054e0 con 0x7fcb64100750 2024-03-19T02:01:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.346+0000 7fcb5bfff640 1 -- 172.21.15.92:0/1030214518 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54003b80 con 0x7fcb64100750 2024-03-19T02:01:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.348+0000 7fcb5bfff640 1 -- 172.21.15.92:0/1030214518 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fcb54046020 con 0x7fcb64100750 2024-03-19T02:01:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.348+0000 7fcb6bd7b740 1 Processor -- start 2024-03-19T02:01:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6bd7b740 1 -- start start 2024-03-19T02:01:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b28520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b2d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b2dc60 0x5573a2b300f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573a2a8cbe0 con 0x5573a2b28100 2024-03-19T02:01:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5573a2a35960 con 0x5573a2b28af0 2024-03-19T02:01:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5573a2a88c80 con 0x5573a2b2dc60 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b28520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b28520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42414/0 (socket says 172.21.15.92:42414) 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6359e640 1 -- 172.21.15.92:0/975821160 learned_addr learned my addr 172.21.15.92:0/975821160 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b2d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6259c640 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b2dc60 0x5573a2b300f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb5bfff640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcb500a28a0 0x7fcb500a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6359e640 1 -- 172.21.15.92:0/975821160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b2dc60 msgr2=0x5573a2b300f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6359e640 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b2dc60 0x5573a2b300f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6359e640 1 -- 172.21.15.92:0/975821160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 msgr2=0x5573a2b2d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6359e640 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b2d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb6359e640 1 -- 172.21.15.92:0/975821160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a2a2cd20 con 0x5573a2b28100 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b2d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.349+0000 7fcb5bfff640 1 -- 172.21.15.92:0/1030214518 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcb540f0360 con 0x7fcb64100750 2024-03-19T02:01:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.350+0000 7fcb6359e640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcb500a28a0 0x7fcb500a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.350+0000 7fcb6359e640 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b28520 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009f50 tx=0x7fcb5c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991715b690cb12d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.350+0000 7fcb597fa640 1 -- 172.21.15.92:0/975821160 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c004250 con 0x5573a2b28100 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.350+0000 7fcb597fa640 1 -- 172.21.15.92:0/975821160 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb5c00fe80 con 0x5573a2b28100 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.350+0000 7fcb597fa640 1 -- 172.21.15.92:0/975821160 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c01bae0 con 0x5573a2b28100 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6359e640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcb500a28a0 0x7fcb500a4d60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c012fd0 tx=0x7fcb5c021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/975821160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 msgr2=0x5573a2b28520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b28520 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009f50 tx=0x7fcb5c0118d0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/975821160 shutdown_connections 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b2dc60 0x5573a2b300f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b2d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/975821160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b28520 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/975821160 >> 172.21.15.92:0/975821160 conn(0x5573a2b23670 msgr2=0x5573a2b23a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/975821160 shutdown_connections 2024-03-19T02:01:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/975821160 wait complete. 2024-03-19T02:01:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 -- start start 2024-03-19T02:01:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 Processor -- start 2024-03-19T02:01:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b28100 0x5573a2b32100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b32640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b2dc60 0x5573a2b39680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5573a2b30c10 con 0x5573a2b2dc60 2024-03-19T02:01:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5573a2a48db0 con 0x5573a2b28af0 2024-03-19T02:01:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5573a2b30d90 con 0x5573a2b28100 2024-03-19T02:01:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.351+0000 7fcb6bd7b740 10 client.0 _ll_register_callbacks cb 0x5573a2a4de70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b2dc60 0x5573a2b39680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb62d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b28100 0x5573a2b32100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b2dc60 0x5573a2b39680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42418/0 (socket says 172.21.15.92:42418) 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b32640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6359e640 1 -- 172.21.15.92:0/1569706569 learned_addr learned my addr 172.21.15.92:0/1569706569 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6359e640 1 -- 172.21.15.92:0/1569706569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b28100 msgr2=0x5573a2b32100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6359e640 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b28100 0x5573a2b32100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6359e640 1 -- 172.21.15.92:0/1569706569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 msgr2=0x5573a2b32640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6359e640 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b32640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6359e640 1 -- 172.21.15.92:0/1569706569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a2a47c60 con 0x5573a2b2dc60 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b28100 0x5573a2b32100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6259c640 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b32640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6359e640 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b2dc60 0x5573a2b39680 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c01bc30 tx=0x7fcb5c003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475e4c79e265976c server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/1569706569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c033030 con 0x5573a2b2dc60 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/1569706569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcb5c004260 con 0x5573a2b2dc60 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/1569706569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5573a2a9a120 con 0x5573a2b2dc60 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/1569706569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb5c009be0 con 0x5573a2b2dc60 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6bd7b740 10 client.16050 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/1569706569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573a27a1210 con 0x5573a2b2dc60 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.352+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/1569706569 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5573a2a9a120 con 0x5573a2b2dc60 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/1569706569 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcb5c038af0 con 0x5573a2b2dc60 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb6bd7b740 10 client.16050 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb6bd7b740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb6bd7b740 20 client.16050 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5573a2b3deb0 0x5573a2b40350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb6bd7b740 4 client.16050 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/1569706569 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573a2a94470 con 0x5573a2b3deb0 2024-03-19T02:01:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5573a2b3deb0 0x5573a2b40350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb4a7fc640 10 client.16050 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.353+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5573a2b3deb0 0x5573a2b40350 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3566be39054b5c server_cookie=1d8daeca0b668601 in_seq=0 out_seq=0 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/1569706569 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5573a2a94470 con 0x5573a2b3deb0 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb4a7fc640 10 client.16050 handle_command_reply: tid=0 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb6bd7b740 1 client.16050 shutdown 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/1569706569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b2dc60 msgr2=0x5573a2b39680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b2dc60 0x5573a2b39680 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c01bc30 tx=0x7fcb5c003a40 comp rx=0 tx=0).stop 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/1569706569 shutdown_connections 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5573a2b3deb0 0x5573a2b40350 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b2dc60 0x5573a2b39680 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b32640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/1569706569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b28100 0x5573a2b32100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/1569706569 >> 172.21.15.92:0/1569706569 conn(0x5573a2b23670 msgr2=0x5573a2b248c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.354+0000 7fcb4a7fc640 0 client.16050 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.355+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/1569706569 shutdown_connections 2024-03-19T02:01:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.355+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/1569706569 wait complete. 2024-03-19T02:01:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.355+0000 7fcb6bd7b740 20 client.16050 trim_cache size 0 max 16384 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.369+0000 7fcb6bd7b740 1 Processor -- start 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.369+0000 7fcb6bd7b740 1 -- start start 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.369+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 0x5573a2b311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.369+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b31700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.369+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.369+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4400e400 con 0x5573a2b28100 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.369+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4400ff40 con 0x5573a2b28af0 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.369+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4400ead0 con 0x5573a2b23670 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb62d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 0x5573a2b311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b31700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42428/0 (socket says 172.21.15.92:42428) 2024-03-19T02:01:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6259c640 1 -- 172.21.15.92:0/2764398210 learned_addr learned my addr 172.21.15.92:0/2764398210 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6359e640 1 --2- 172.21.15.92:0/2764398210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6259c640 1 -- 172.21.15.92:0/2764398210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 msgr2=0x5573a2b311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6259c640 1 --2- 172.21.15.92:0/2764398210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 0x5573a2b311c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6259c640 1 -- 172.21.15.92:0/2764398210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 msgr2=0x5573a2b39640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6259c640 1 --2- 172.21.15.92:0/2764398210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b39640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6259c640 1 -- 172.21.15.92:0/2764398210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a286d620 con 0x5573a2b28100 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb6259c640 1 --2- 172.21.15.92:0/2764398210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b31700 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c012580 tx=0x7fcb4c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6d030ef414e661 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb58ff9640 1 -- 172.21.15.92:0/2764398210 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c022070 con 0x5573a2b28100 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb58ff9640 1 -- 172.21.15.92:0/2764398210 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4c0035f0 con 0x5573a2b28100 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.370+0000 7fcb58ff9640 1 -- 172.21.15.92:0/2764398210 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c01d6c0 con 0x5573a2b28100 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/2764398210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 msgr2=0x5573a2b31700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/2764398210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b31700 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c012580 tx=0x7fcb4c005f90 comp rx=0 tx=0).stop 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/2764398210 shutdown_connections 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/2764398210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28af0 0x5573a2b39640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/2764398210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28100 0x5573a2b31700 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/2764398210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 0x5573a2b311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/2764398210 >> 172.21.15.92:0/2764398210 conn(0x5573a2b1ecd0 msgr2=0x5573a2b2d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/2764398210 shutdown_connections 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/2764398210 wait complete. 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 -- start start 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 Processor -- start 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 0x5573a2b395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28100 0x5573a2b36050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28af0 0x5573a2b36610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4400f870 con 0x5573a2b28af0 2024-03-19T02:01:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4400f1a0 con 0x5573a2b28100 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.371+0000 7fcb6bd7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4400c3e0 con 0x5573a2b23670 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6bd7b740 10 client.0 _ll_register_callbacks cb 0x7fcb540f3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28af0 0x5573a2b36610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb62d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 0x5573a2b395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28af0 0x5573a2b36610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42444/0 (socket says 172.21.15.92:42444) 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6359e640 1 -- 172.21.15.92:0/3129607525 learned_addr learned my addr 172.21.15.92:0/3129607525 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6259c640 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28100 0x5573a2b36050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6359e640 1 -- 172.21.15.92:0/3129607525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 msgr2=0x5573a2b395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6359e640 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 0x5573a2b395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6359e640 1 -- 172.21.15.92:0/3129607525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28100 msgr2=0x5573a2b36050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6359e640 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28100 0x5573a2b36050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6359e640 1 -- 172.21.15.92:0/3129607525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a2ba2270 con 0x5573a2b28af0 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6359e640 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28af0 0x5573a2b36610 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c002230 tx=0x7fcb5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee26fdfecb6b754 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:01:53.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/3129607525 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c009b30 con 0x5573a2b28af0 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/3129607525 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcb5c034040 con 0x5573a2b28af0 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/3129607525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb4c0035f0 con 0x5573a2b28af0 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/3129607525 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcb5c052050 con 0x5573a2b28af0 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6bd7b740 10 client.16056 fetch_fsmap learned FSMap version 29 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.372+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/3129607525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573a2ba3580 con 0x5573a2b28af0 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/3129607525 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4c0035f0 con 0x5573a2b28af0 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/3129607525 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcb5c038450 con 0x5573a2b28af0 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb6bd7b740 10 client.16056 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb6bd7b740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb6bd7b740 20 client.16056 populate_metadata read hostname 'smithi092' 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5573a2b2dc60 0x5573a2ba5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb6bd7b740 4 client.16056 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/3129607525 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573a2a12770 con 0x5573a2b2dc60 2024-03-19T02:01:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5573a2b2dc60 0x5573a2ba5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:01:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb4a7fc640 10 client.16056 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.373+0000 7fcb62d9d640 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5573a2b2dc60 0x5573a2ba5eb0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389991c771fdf786 server_cookie=ba7f9f87b78981ce in_seq=0 out_seq=0 2024-03-19T02:01:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb4a7fc640 1 -- 172.21.15.92:0/3129607525 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573a2a12770 con 0x5573a2b2dc60 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb4a7fc640 10 client.16056 handle_command_reply: tid=0 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 client.16056 shutdown 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/3129607525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28af0 msgr2=0x5573a2b36610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28af0 0x5573a2b36610 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c002230 tx=0x7fcb5c004a40 comp rx=0 tx=0).stop 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/3129607525 shutdown_connections 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5573a2b2dc60 0x5573a2ba5eb0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5573a2b28af0 0x5573a2b36610 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573a2b28100 0x5573a2b36050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb4a7fc640 0 client.16056 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 --2- 172.21.15.92:0/3129607525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573a2b23670 0x5573a2b395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/3129607525 >> 172.21.15.92:0/3129607525 conn(0x5573a2b1ecd0 msgr2=0x5573a2b320c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/3129607525 shutdown_connections 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 1 -- 172.21.15.92:0/3129607525 wait complete. 2024-03-19T02:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb6bd7b740 20 client.16056 trim_cache size 0 max 16384 2024-03-19T02:01:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:01:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:01:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:01:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.380+0000 7fcb69189640 1 -- 172.21.15.92:0/1030214518 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcb500a28a0 msgr2=0x7fcb500a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcb500a28a0 0x7fcb500a4d60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c012fd0 tx=0x7fcb5c021000 comp rx=0 tx=0).stop 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 -- 172.21.15.92:0/1030214518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb64100750 msgr2=0x7fcb6406abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb64100750 0x7fcb6406abd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009230 tx=0x7fcb5402c5d0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 -- 172.21.15.92:0/1030214518 shutdown_connections 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb64102300 0x7fcb6419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcb500a28a0 0x7fcb500a4d60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb64101940 0x7fcb6419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 --2- 172.21.15.92:0/1030214518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb64100750 0x7fcb6406abd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 -- 172.21.15.92:0/1030214518 >> 172.21.15.92:0/1030214518 conn(0x7fcb640fbcc0 msgr2=0x7fcb64104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:01:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 -- 172.21.15.92:0/1030214518 shutdown_connections 2024-03-19T02:01:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:01:53.381+0000 7fcb69189640 1 -- 172.21.15.92:0/1030214518 wait complete. 2024-03-19T02:01:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:54 smithi159 ceph-mon[67110]: pgmap v624: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:01:55.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:54 smithi092 ceph-mon[62574]: pgmap v624: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:01:55.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:54 smithi144 ceph-mon[67199]: pgmap v624: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:01:56.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:56 smithi159 ceph-mon[67110]: pgmap v625: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.0 KiB/s wr, 13 op/s 2024-03-19T02:01:57.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:56 smithi092 ceph-mon[62574]: pgmap v625: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.0 KiB/s wr, 13 op/s 2024-03-19T02:01:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:56 smithi144 ceph-mon[67199]: pgmap v625: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.0 KiB/s wr, 13 op/s 2024-03-19T02:01:58.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:01:58 smithi159 ceph-mon[67110]: pgmap v626: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:01:58 smithi092 ceph-mon[62574]: pgmap v626: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:01:58 smithi144 ceph-mon[67199]: pgmap v626: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:01:59.996 INFO:tasks.workunit.client.0.smithi092.stdout:437/600: open fd = 3 2024-03-19T02:01:59.996 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.997 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.997 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:01:59.997 INFO:tasks.workunit.client.0.smithi092.stdout:438/600: open fd = 3 2024-03-19T02:01:59.997 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.997 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.997 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:01:59.997 INFO:tasks.workunit.client.0.smithi092.stdout:439/600: open fd = 3 2024-03-19T02:01:59.998 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.998 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.998 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:01:59.998 INFO:tasks.workunit.client.0.smithi092.stdout:440/600: open fd = 3 2024-03-19T02:01:59.998 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.998 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.999 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:01:59.999 INFO:tasks.workunit.client.0.smithi092.stdout:441/600: open fd = 3 2024-03-19T02:01:59.999 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.999 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:01:59.999 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:01:59.999 INFO:tasks.workunit.client.0.smithi092.stdout:442/600: open fd = 3 2024-03-19T02:02:00.000 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.000 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.000 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.000 INFO:tasks.workunit.client.0.smithi092.stdout:443/600: open fd = 3 2024-03-19T02:02:00.000 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.000 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.001 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.001 INFO:tasks.workunit.client.0.smithi092.stdout:444/600: open fd = 3 2024-03-19T02:02:00.001 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.001 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.001 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.001 INFO:tasks.workunit.client.0.smithi092.stdout:445/600: open fd = 3 2024-03-19T02:02:00.002 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.002 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.002 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.002 INFO:tasks.workunit.client.0.smithi092.stdout:446/600: open fd = 3 2024-03-19T02:02:00.002 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.002 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.003 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.003 INFO:tasks.workunit.client.0.smithi092.stdout:447/600: open fd = 3 2024-03-19T02:02:00.003 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.003 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.003 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.003 INFO:tasks.workunit.client.0.smithi092.stdout:448/600: open fd = 3 2024-03-19T02:02:00.003 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.004 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.004 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.004 INFO:tasks.workunit.client.0.smithi092.stdout:449/600: open fd = 3 2024-03-19T02:02:00.004 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.004 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.004 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.005 INFO:tasks.workunit.client.0.smithi092.stdout:450/600: open fd = 3 2024-03-19T02:02:00.005 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.005 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.005 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.005 INFO:tasks.workunit.client.0.smithi092.stdout:451/600: open fd = 3 2024-03-19T02:02:00.005 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.005 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.006 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.006 INFO:tasks.workunit.client.0.smithi092.stdout:452/600: open fd = 3 2024-03-19T02:02:00.006 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.006 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.006 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.006 INFO:tasks.workunit.client.0.smithi092.stdout:453/600: open fd = 3 2024-03-19T02:02:00.007 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.007 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.007 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.007 INFO:tasks.workunit.client.0.smithi092.stdout:454/600: open fd = 3 2024-03-19T02:02:00.007 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.007 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.008 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.008 INFO:tasks.workunit.client.0.smithi092.stdout:455/600: open fd = 3 2024-03-19T02:02:00.008 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.008 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.008 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.008 INFO:tasks.workunit.client.0.smithi092.stdout:456/600: open fd = 3 2024-03-19T02:02:00.008 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.009 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.009 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.009 INFO:tasks.workunit.client.0.smithi092.stdout:457/600: open fd = 3 2024-03-19T02:02:00.009 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.009 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.009 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.009 INFO:tasks.workunit.client.0.smithi092.stdout:458/600: open fd = 3 2024-03-19T02:02:00.010 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.010 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.010 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.010 INFO:tasks.workunit.client.0.smithi092.stdout:459/600: open fd = 3 2024-03-19T02:02:00.010 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.010 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.011 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.011 INFO:tasks.workunit.client.0.smithi092.stdout:460/600: open fd = 3 2024-03-19T02:02:00.011 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.011 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.011 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.011 INFO:tasks.workunit.client.0.smithi092.stdout:461/600: open fd = 3 2024-03-19T02:02:00.012 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.012 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.012 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.012 INFO:tasks.workunit.client.0.smithi092.stdout:462/600: open fd = 3 2024-03-19T02:02:00.012 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.012 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.012 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.013 INFO:tasks.workunit.client.0.smithi092.stdout:463/600: open fd = 3 2024-03-19T02:02:00.013 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.013 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.013 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.013 INFO:tasks.workunit.client.0.smithi092.stdout:464/600: open fd = 3 2024-03-19T02:02:00.013 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.014 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.014 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.014 INFO:tasks.workunit.client.0.smithi092.stdout:465/600: open fd = 3 2024-03-19T02:02:00.014 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.014 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.014 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.015 INFO:tasks.workunit.client.0.smithi092.stdout:466/600: open fd = 3 2024-03-19T02:02:00.015 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.015 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.015 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.015 INFO:tasks.workunit.client.0.smithi092.stdout:467/600: open fd = 3 2024-03-19T02:02:00.015 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.015 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.016 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.016 INFO:tasks.workunit.client.0.smithi092.stdout:468/600: open fd = 3 2024-03-19T02:02:00.016 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.016 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.016 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.016 INFO:tasks.workunit.client.0.smithi092.stdout:469/600: open fd = 3 2024-03-19T02:02:00.016 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.017 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.017 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.017 INFO:tasks.workunit.client.0.smithi092.stdout:470/600: open fd = 3 2024-03-19T02:02:00.017 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.017 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.017 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.017 INFO:tasks.workunit.client.0.smithi092.stdout:471/600: open fd = 3 2024-03-19T02:02:00.017 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.017 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.018 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.018 INFO:tasks.workunit.client.0.smithi092.stdout:472/600: open fd = 3 2024-03-19T02:02:00.018 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.018 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.018 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.018 INFO:tasks.workunit.client.0.smithi092.stdout:473/600: open fd = 3 2024-03-19T02:02:00.018 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.018 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.018 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.019 INFO:tasks.workunit.client.0.smithi092.stdout:474/600: open fd = 3 2024-03-19T02:02:00.019 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.019 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.019 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.019 INFO:tasks.workunit.client.0.smithi092.stdout:475/600: open fd = 3 2024-03-19T02:02:00.019 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.019 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.019 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.020 INFO:tasks.workunit.client.0.smithi092.stdout:476/600: open fd = 3 2024-03-19T02:02:00.020 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.020 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.020 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.020 INFO:tasks.workunit.client.0.smithi092.stdout:477/600: open fd = 3 2024-03-19T02:02:00.020 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.020 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.020 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:478/600: open fd = 3 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:479/600: open fd = 3 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:480/600: open fd = 3 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:481/600: open fd = 3 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.021 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:482/600: open fd = 3 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:483/600: open fd = 3 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:484/600: open fd = 3 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.022 INFO:tasks.workunit.client.0.smithi092.stdout:485/600: open fd = 3 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:486/600: open fd = 3 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:487/600: open fd = 3 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.023 INFO:tasks.workunit.client.0.smithi092.stdout:488/600: open fd = 3 2024-03-19T02:02:00.024 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.025 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.025 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.025 INFO:tasks.workunit.client.0.smithi092.stdout:489/600: open fd = 3 2024-03-19T02:02:00.025 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.025 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.025 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.026 INFO:tasks.workunit.client.0.smithi092.stdout:490/600: open fd = 3 2024-03-19T02:02:00.026 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.026 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.026 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.026 INFO:tasks.workunit.client.0.smithi092.stdout:491/600: open fd = 3 2024-03-19T02:02:00.027 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.027 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.027 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.027 INFO:tasks.workunit.client.0.smithi092.stdout:492/600: open fd = 3 2024-03-19T02:02:00.027 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.027 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.028 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.028 INFO:tasks.workunit.client.0.smithi092.stdout:493/600: open fd = 3 2024-03-19T02:02:00.028 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.028 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.028 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.028 INFO:tasks.workunit.client.0.smithi092.stdout:494/600: open fd = 3 2024-03-19T02:02:00.029 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.029 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.029 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.029 INFO:tasks.workunit.client.0.smithi092.stdout:495/600: open fd = 3 2024-03-19T02:02:00.029 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.030 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.030 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.030 INFO:tasks.workunit.client.0.smithi092.stdout:496/600: open fd = 3 2024-03-19T02:02:00.030 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.030 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.031 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.031 INFO:tasks.workunit.client.0.smithi092.stdout:497/600: open fd = 3 2024-03-19T02:02:00.031 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.031 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.031 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.031 INFO:tasks.workunit.client.0.smithi092.stdout:498/600: open fd = 3 2024-03-19T02:02:00.032 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.032 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:02:00.032 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:02:00.958 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:00 smithi159 ceph-mon[67110]: pgmap v627: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:00 smithi092 ceph-mon[62574]: pgmap v627: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:00 smithi144 ceph-mon[67199]: pgmap v627: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:02.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:02 smithi159 ceph-mon[67110]: pgmap v628: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:02:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:02 smithi092 ceph-mon[62574]: pgmap v628: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:02:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:02 smithi144 ceph-mon[67199]: pgmap v628: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:02:04.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:04 smithi159 ceph-mon[67110]: pgmap v629: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:02:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:04 smithi092 ceph-mon[62574]: pgmap v629: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:02:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:04 smithi144 ceph-mon[67199]: pgmap v629: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:02:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.469+0000 7f48345b2640 1 -- 172.21.15.92:0/2379213520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c100750 msgr2=0x7f482c100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.469+0000 7f48345b2640 1 --2- 172.21.15.92:0/2379213520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c100750 0x7f482c100b70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f481c009900 tx=0x7f481c02bc30 comp rx=0 tx=0).stop 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.469+0000 7f48345b2640 1 -- 172.21.15.92:0/2379213520 shutdown_connections 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.469+0000 7f48345b2640 1 --2- 172.21.15.92:0/2379213520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f482c102300 0x7f482c108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.469+0000 7f48345b2640 1 --2- 172.21.15.92:0/2379213520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101940 0x7f482c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.469+0000 7f48345b2640 1 --2- 172.21.15.92:0/2379213520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c100750 0x7f482c100b70 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.469+0000 7f48345b2640 1 -- 172.21.15.92:0/2379213520 >> 172.21.15.92:0/2379213520 conn(0x7f482c0fbcc0 msgr2=0x7f482c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.469+0000 7f48345b2640 1 -- 172.21.15.92:0/2379213520 shutdown_connections 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.469+0000 7f48345b2640 1 -- 172.21.15.92:0/2379213520 wait complete. 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f48345b2640 1 Processor -- start 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f48345b2640 1 -- start start 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f48345b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f482c100750 0x7f482c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f48345b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101940 0x7f482c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f48345b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c102300 0x7f482c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f48345b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10d4f0 con 0x7f482c102300 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f48345b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10d370 con 0x7f482c101940 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f48345b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10d670 con 0x7f482c100750 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f4832b4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c102300 0x7f482c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f4832b4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c102300 0x7f482c19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40764/0 (socket says 172.21.15.92:40764) 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f4832b4f640 1 -- 172.21.15.92:0/2652244035 learned_addr learned my addr 172.21.15.92:0/2652244035 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f483234e640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f482c100750 0x7f482c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f4831b4d640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101940 0x7f482c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f4832b4f640 1 -- 172.21.15.92:0/2652244035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f482c100750 msgr2=0x7f482c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f4832b4f640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f482c100750 0x7f482c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f4832b4f640 1 -- 172.21.15.92:0/2652244035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101940 msgr2=0x7f482c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f4832b4f640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101940 0x7f482c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f4832b4f640 1 -- 172.21.15.92:0/2652244035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c19f760 con 0x7f482c102300 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.470+0000 7f483234e640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f482c100750 0x7f482c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.471+0000 7f4832b4f640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c102300 0x7f482c19f0f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f482400eb10 tx=0x7f482400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.471+0000 7f48237fe640 1 -- 172.21.15.92:0/2652244035 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824004b20 con 0x7f482c102300 2024-03-19T02:02:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.471+0000 7f48237fe640 1 -- 172.21.15.92:0/2652244035 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824004cc0 con 0x7f482c102300 2024-03-19T02:02:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.471+0000 7f48345b2640 1 -- 172.21.15.92:0/2652244035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c19fa50 con 0x7f482c102300 2024-03-19T02:02:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.471+0000 7f48237fe640 1 -- 172.21.15.92:0/2652244035 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824010760 con 0x7f482c102300 2024-03-19T02:02:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.471+0000 7f48345b2640 1 -- 172.21.15.92:0/2652244035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c19ffa0 con 0x7f482c102300 2024-03-19T02:02:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.473+0000 7f48237fe640 1 -- 172.21.15.92:0/2652244035 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f4824005020 con 0x7f482c102300 2024-03-19T02:02:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.473+0000 7f483719f740 1 Processor -- start 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483719f740 1 -- start start 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483719f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7edc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483719f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edcf00 0x5636c7ee1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483719f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7ee4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483719f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5636c7e3cbe0 con 0x5636c7edc510 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483719f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5636c7de5960 con 0x5636c7edcf00 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483719f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636c7e38c80 con 0x5636c7ee2070 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f48237fe640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48180a28a0 0x7f48180a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f4831b4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7edc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483234e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edcf00 0x5636c7ee1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483234e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edcf00 0x5636c7ee1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:60028/0 (socket says 172.21.15.92:60028) 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483234e640 1 -- 172.21.15.92:0/191016151 learned_addr learned my addr 172.21.15.92:0/191016151 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f4831b4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7edc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40776/0 (socket says 172.21.15.92:40776) 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f4831b4d640 1 --2- 172.21.15.92:0/191016151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7ee4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f4832b4f640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48180a28a0 0x7f48180a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f48237fe640 1 -- 172.21.15.92:0/2652244035 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f48240cefd0 con 0x7f482c102300 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483234e640 1 -- 172.21.15.92:0/191016151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 msgr2=0x5636c7ee4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483234e640 1 --2- 172.21.15.92:0/191016151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7ee4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483234e640 1 -- 172.21.15.92:0/191016151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 msgr2=0x5636c7edc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483234e640 1 --2- 172.21.15.92:0/191016151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7edc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f483234e640 1 -- 172.21.15.92:0/191016151 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c7ddcd20 con 0x5636c7edcf00 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.474+0000 7f4831b4d640 1 --2- 172.21.15.92:0/191016151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7edc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.475+0000 7f4832b4f640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48180a28a0 0x7f48180a4d60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4824021000 tx=0x7f482409e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.475+0000 7f483234e640 1 --2- 172.21.15.92:0/191016151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edcf00 0x5636c7ee1a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f481c009900 tx=0x7f481c02cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b8727f73e9b6e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.475+0000 7f4820bfe640 1 -- 172.21.15.92:0/191016151 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c005d50 con 0x5636c7edcf00 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.475+0000 7f4820bfe640 1 -- 172.21.15.92:0/191016151 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481c005520 con 0x5636c7edcf00 2024-03-19T02:02:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.475+0000 7f4820bfe640 1 -- 172.21.15.92:0/191016151 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c004b20 con 0x5636c7edcf00 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.475+0000 7f483719f740 1 -- 172.21.15.92:0/191016151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edcf00 msgr2=0x5636c7ee1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.475+0000 7f483719f740 1 --2- 172.21.15.92:0/191016151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edcf00 0x5636c7ee1a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f481c009900 tx=0x7f481c02cd30 comp rx=0 tx=0).stop 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 -- 172.21.15.92:0/191016151 shutdown_connections 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 --2- 172.21.15.92:0/191016151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7ee4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 --2- 172.21.15.92:0/191016151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edcf00 0x5636c7ee1a00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 --2- 172.21.15.92:0/191016151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7edc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 -- 172.21.15.92:0/191016151 >> 172.21.15.92:0/191016151 conn(0x5636c7ed3670 msgr2=0x5636c7ed3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 -- 172.21.15.92:0/191016151 shutdown_connections 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 -- 172.21.15.92:0/191016151 wait complete. 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 -- start start 2024-03-19T02:02:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 Processor -- start 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 0x5636c7ee6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7ee69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7eeda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5636c7ee4fc0 con 0x5636c7edcf00 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5636c7df8db0 con 0x5636c7edc510 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636c7ee5140 con 0x5636c7ee2070 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483719f740 10 client.0 _ll_register_callbacks cb 0x5636c7dfde70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483234e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7ee69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483234e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7ee69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40792/0 (socket says 172.21.15.92:40792) 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f483234e640 1 -- 172.21.15.92:0/3837935935 learned_addr learned my addr 172.21.15.92:0/3837935935 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f4831b4d640 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 0x5636c7ee6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.476+0000 7f4831b4d640 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7eeda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f483234e640 1 -- 172.21.15.92:0/3837935935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 msgr2=0x5636c7eeda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f483234e640 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7eeda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f483234e640 1 -- 172.21.15.92:0/3837935935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 msgr2=0x5636c7ee6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f483234e640 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 0x5636c7ee6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f483234e640 1 -- 172.21.15.92:0/3837935935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c7df7c60 con 0x5636c7edcf00 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f4831b4d640 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7eeda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f483234e640 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7ee69c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f481c036040 tx=0x7f481c034ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda1ba1192f9cee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f4815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f4815ffb640 1 -- 172.21.15.92:0/3837935935 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c005d50 con 0x5636c7edcf00 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f4815ffb640 1 -- 172.21.15.92:0/3837935935 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f481c045620 con 0x5636c7edcf00 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f483719f740 1 -- 172.21.15.92:0/3837935935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f481c005520 con 0x5636c7edcf00 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f4815ffb640 1 -- 172.21.15.92:0/3837935935 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f481c045820 con 0x5636c7edcf00 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f483719f740 10 client.16068 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f483719f740 1 -- 172.21.15.92:0/3837935935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636c7b51210 con 0x5636c7edcf00 2024-03-19T02:02:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f4815ffb640 1 -- 172.21.15.92:0/3837935935 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481c005520 con 0x5636c7edcf00 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.477+0000 7f4815ffb640 1 -- 172.21.15.92:0/3837935935 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f481c04e860 con 0x5636c7edcf00 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.478+0000 7f483719f740 10 client.16068 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.478+0000 7f483719f740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.478+0000 7f483719f740 20 client.16068 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.478+0000 7f483719f740 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5636c7ef1ef0 0x5636c7ef42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.478+0000 7f483719f740 4 client.16068 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.478+0000 7f483719f740 1 -- 172.21.15.92:0/3837935935 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636c7e4a120 con 0x5636c7ef1ef0 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.478+0000 7f4831b4d640 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5636c7ef1ef0 0x5636c7ef42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.478+0000 7f4815ffb640 10 client.16068 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.478+0000 7f4831b4d640 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5636c7ef1ef0 0x5636c7ef42e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f200e2706a9ba8 server_cookie=cd04560f7b4e31c4 in_seq=0 out_seq=0 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f4815ffb640 1 -- 172.21.15.92:0/3837935935 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5636c7e4a120 con 0x5636c7ef1ef0 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f4815ffb640 10 client.16068 handle_command_reply: tid=0 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 client.16068 shutdown 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 -- 172.21.15.92:0/3837935935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 msgr2=0x5636c7ee69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7ee69c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f481c036040 tx=0x7f481c034ac0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 -- 172.21.15.92:0/3837935935 shutdown_connections 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5636c7ef1ef0 0x5636c7ef42e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7eeda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7ee69c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f4815ffb640 0 client.16068 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 --2- 172.21.15.92:0/3837935935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 0x5636c7ee6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 -- 172.21.15.92:0/3837935935 >> 172.21.15.92:0/3837935935 conn(0x5636c7ed3670 msgr2=0x5636c7ed48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 -- 172.21.15.92:0/3837935935 shutdown_connections 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 1 -- 172.21.15.92:0/3837935935 wait complete. 2024-03-19T02:02:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.479+0000 7f483719f740 20 client.16068 trim_cache size 0 max 16384 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f483719f740 1 Processor -- start 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f483719f740 1 -- start start 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f483719f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7ed6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f483719f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7edcf00 0x5636c7eed9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f483719f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7ee2070 0x5636c7ef8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f483719f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1a6a90 con 0x5636c7edc510 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f483719f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1a85d0 con 0x5636c7ee2070 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f483719f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1a7160 con 0x5636c7edcf00 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7ed6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7ed6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40804/0 (socket says 172.21.15.92:40804) 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 -- 172.21.15.92:0/413826037 learned_addr learned my addr 172.21.15.92:0/413826037 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f483234e640 1 --2- 172.21.15.92:0/413826037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7edcf00 0x5636c7eed9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 --2- 172.21.15.92:0/413826037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7ee2070 0x5636c7ef8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 -- 172.21.15.92:0/413826037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7edcf00 msgr2=0x5636c7eed9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 --2- 172.21.15.92:0/413826037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7edcf00 0x5636c7eed9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 -- 172.21.15.92:0/413826037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7ee2070 msgr2=0x5636c7ef8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 --2- 172.21.15.92:0/413826037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7ee2070 0x5636c7ef8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 -- 172.21.15.92:0/413826037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c7c1d620 con 0x5636c7edc510 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4831b4d640 1 --2- 172.21.15.92:0/413826037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7ed6fe0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4828005d00 tx=0x7f48280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e639ba06189bde server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.494+0000 7f4817fff640 1 -- 172.21.15.92:0/413826037 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482801a040 con 0x5636c7edc510 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f4817fff640 1 -- 172.21.15.92:0/413826037 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48280047a0 con 0x5636c7edc510 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f4817fff640 1 -- 172.21.15.92:0/413826037 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828003910 con 0x5636c7edc510 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 -- 172.21.15.92:0/413826037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 msgr2=0x5636c7ed6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 --2- 172.21.15.92:0/413826037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7ed6fe0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4828005d00 tx=0x7f48280046c0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 -- 172.21.15.92:0/413826037 shutdown_connections 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 --2- 172.21.15.92:0/413826037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7ee2070 0x5636c7ef8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 --2- 172.21.15.92:0/413826037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7edcf00 0x5636c7eed9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 --2- 172.21.15.92:0/413826037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edc510 0x5636c7ed6fe0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 -- 172.21.15.92:0/413826037 >> 172.21.15.92:0/413826037 conn(0x5636c7ed3670 msgr2=0x5636c7ed68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 -- 172.21.15.92:0/413826037 shutdown_connections 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 -- 172.21.15.92:0/413826037 wait complete. 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 -- start start 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.495+0000 7f483719f740 1 Processor -- start 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 0x5636c7eed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7eebce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7eec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1a7f00 con 0x5636c7edcf00 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1a7830 con 0x5636c7edc510 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1a4a70 con 0x5636c7ee2070 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 10 client.0 _ll_register_callbacks cb 0x7f4828009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483234e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7eebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483234e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7eebce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40818/0 (socket says 172.21.15.92:40818) 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483234e640 1 -- 172.21.15.92:0/92287687 learned_addr learned my addr 172.21.15.92:0/92287687 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f4831b4d640 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7eec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f4831b4d640 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 0x5636c7eed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483234e640 1 -- 172.21.15.92:0/92287687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 msgr2=0x5636c7eec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483234e640 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7eec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483234e640 1 -- 172.21.15.92:0/92287687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 msgr2=0x5636c7eed660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483234e640 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 0x5636c7eed660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483234e640 1 -- 172.21.15.92:0/92287687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c7eec7e0 con 0x5636c7edcf00 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f4831b4d640 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 0x5636c7eed660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483234e640 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7eebce0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f481c034860 tx=0x7f481c02c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da30adcefb069b22 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f4815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f4815ffb640 1 -- 172.21.15.92:0/92287687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c005d50 con 0x5636c7edcf00 2024-03-19T02:02:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f4815ffb640 1 -- 172.21.15.92:0/92287687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f481c045dd0 con 0x5636c7edcf00 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 1 -- 172.21.15.92:0/92287687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48280047a0 con 0x5636c7edcf00 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f4815ffb640 1 -- 172.21.15.92:0/92287687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f481c004020 con 0x5636c7edcf00 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 10 client.16074 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.496+0000 7f483719f740 1 -- 172.21.15.92:0/92287687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636c7f58fc0 con 0x5636c7edcf00 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f4815ffb640 1 -- 172.21.15.92:0/92287687 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48280047a0 con 0x5636c7edcf00 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f4815ffb640 1 -- 172.21.15.92:0/92287687 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f481c005070 con 0x5636c7edcf00 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f483719f740 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f483719f740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f483719f740 20 client.16074 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f483719f740 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5636c7f59910 0x5636c7f5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f483719f740 4 client.16074 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f483719f740 1 -- 172.21.15.92:0/92287687 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636c7e44470 con 0x5636c7f59910 2024-03-19T02:02:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f4831b4d640 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5636c7f59910 0x5636c7f5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f4815ffb640 10 client.16074 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.497+0000 7f4831b4d640 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5636c7f59910 0x5636c7f5bd30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229b2412e44e8ba4 server_cookie=4613aefb543c5408 in_seq=0 out_seq=0 2024-03-19T02:02:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f4815ffb640 1 -- 172.21.15.92:0/92287687 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636c7e44470 con 0x5636c7f59910 2024-03-19T02:02:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f4815ffb640 10 client.16074 handle_command_reply: tid=0 2024-03-19T02:02:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f483719f740 1 client.16074 shutdown 2024-03-19T02:02:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f483719f740 1 -- 172.21.15.92:0/92287687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 msgr2=0x5636c7eebce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f483719f740 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7eebce0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f481c034860 tx=0x7f481c02c330 comp rx=0 tx=0).stop 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f483719f740 1 -- 172.21.15.92:0/92287687 shutdown_connections 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f483719f740 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5636c7f59910 0x5636c7f5bd30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f483719f740 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636c7ee2070 0x5636c7eec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f4815ffb640 0 client.16074 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f483719f740 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5636c7edcf00 0x5636c7eebce0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f483719f740 1 --2- 172.21.15.92:0/92287687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636c7edc510 0x5636c7eed660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.504+0000 7f483719f740 1 -- 172.21.15.92:0/92287687 >> 172.21.15.92:0/92287687 conn(0x5636c7ed3670 msgr2=0x5636c7ed7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f483719f740 1 -- 172.21.15.92:0/92287687 shutdown_connections 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f483719f740 1 -- 172.21.15.92:0/92287687 wait complete. 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f483719f740 20 client.16074 trim_cache size 0 max 16384 2024-03-19T02:02:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:02:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:02:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:02:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:02:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 -- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48180a28a0 msgr2=0x7f48180a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48180a28a0 0x7f48180a4d60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4824021000 tx=0x7f482409e040 comp rx=0 tx=0).stop 2024-03-19T02:02:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 -- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c102300 msgr2=0x7f482c19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c102300 0x7f482c19f0f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f482400eb10 tx=0x7f482400c2d0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 -- 172.21.15.92:0/2652244035 shutdown_connections 2024-03-19T02:02:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f482c102300 0x7f482c19f0f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482c101940 0x7f482c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f48180a28a0 0x7f48180a4d60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 --2- 172.21.15.92:0/2652244035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f482c100750 0x7f482c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 -- 172.21.15.92:0/2652244035 >> 172.21.15.92:0/2652244035 conn(0x7f482c0fbcc0 msgr2=0x7f482c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 -- 172.21.15.92:0/2652244035 shutdown_connections 2024-03-19T02:02:06.509 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:06.505+0000 7f48345b2640 1 -- 172.21.15.92:0/2652244035 wait complete. 2024-03-19T02:02:06.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:06 smithi159 ceph-mon[67110]: pgmap v630: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T02:02:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:06 smithi092 ceph-mon[62574]: pgmap v630: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T02:02:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:06 smithi144 ceph-mon[67199]: pgmap v630: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-03-19T02:02:08.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:08 smithi159 ceph-mon[67110]: pgmap v631: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T02:02:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:08 smithi092 ceph-mon[62574]: pgmap v631: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T02:02:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:08 smithi144 ceph-mon[67199]: pgmap v631: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-03-19T02:02:10.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:10 smithi159 ceph-mon[67110]: pgmap v632: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T02:02:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:10 smithi092 ceph-mon[62574]: pgmap v632: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T02:02:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:10 smithi144 ceph-mon[67199]: pgmap v632: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 19 op/s 2024-03-19T02:02:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:12 smithi159 ceph-mon[67110]: pgmap v633: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:02:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:12 smithi092 ceph-mon[62574]: pgmap v633: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:02:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:12 smithi144 ceph-mon[67199]: pgmap v633: 97 pgs: 97 active+clean; 3.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:02:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:14 smithi159 ceph-mon[67110]: pgmap v634: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:02:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:14 smithi092 ceph-mon[62574]: pgmap v634: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:02:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:14 smithi144 ceph-mon[67199]: pgmap v634: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:02:16.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:16 smithi159 ceph-mon[67110]: pgmap v635: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:02:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:16 smithi092 ceph-mon[62574]: pgmap v635: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:02:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:16 smithi144 ceph-mon[67199]: pgmap v635: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:02:18.957 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:18 smithi159 ceph-mon[67110]: pgmap v636: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:02:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:18 smithi092 ceph-mon[62574]: pgmap v636: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:02:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:18 smithi144 ceph-mon[67199]: pgmap v636: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:02:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.593+0000 7f054911b640 1 -- 172.21.15.92:0/1273008880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05440ffe70 msgr2=0x7f0544100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.593+0000 7f054911b640 1 --2- 172.21.15.92:0/1273008880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05440ffe70 0x7f0544100290 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0534009900 tx=0x7f053402bc30 comp rx=0 tx=0).stop 2024-03-19T02:02:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.593+0000 7f054911b640 1 -- 172.21.15.92:0/1273008880 shutdown_connections 2024-03-19T02:02:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.593+0000 7f054911b640 1 --2- 172.21.15.92:0/1273008880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0544101a10 0x7f0544108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.593+0000 7f054911b640 1 --2- 172.21.15.92:0/1273008880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0544101050 0x7f05441014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.593+0000 7f054911b640 1 --2- 172.21.15.92:0/1273008880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05440ffe70 0x7f0544100290 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.593+0000 7f054911b640 1 -- 172.21.15.92:0/1273008880 >> 172.21.15.92:0/1273008880 conn(0x7f05440fb470 msgr2=0x7f05440fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.593+0000 7f054911b640 1 -- 172.21.15.92:0/1273008880 shutdown_connections 2024-03-19T02:02:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.593+0000 7f054911b640 1 -- 172.21.15.92:0/1273008880 wait complete. 2024-03-19T02:02:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054911b640 1 Processor -- start 2024-03-19T02:02:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054911b640 1 -- start start 2024-03-19T02:02:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054911b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05440ffe70 0x7f0544197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054911b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0544101050 0x7f0544198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054911b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0544101a10 0x7f054419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054911b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f054410d500 con 0x7f0544101a10 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054911b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f054410d380 con 0x7f05440ffe70 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054911b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f054410d680 con 0x7f0544101050 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0544101a10 0x7f054419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0544101050 0x7f0544198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0544101a10 0x7f054419f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49140/0 (socket says 172.21.15.92:49140) 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054359e640 1 -- 172.21.15.92:0/850202645 learned_addr learned my addr 172.21.15.92:0/850202645 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f054259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0544101050 0x7f0544198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:54278/0 (socket says 172.21.15.92:54278) 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.594+0000 7f0542d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05440ffe70 0x7f0544197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f054259c640 1 -- 172.21.15.92:0/850202645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05440ffe70 msgr2=0x7f0544197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f054259c640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05440ffe70 0x7f0544197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f054259c640 1 -- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0544101a10 msgr2=0x7f054419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f054259c640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0544101a10 0x7f054419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f054259c640 1 -- 172.21.15.92:0/850202645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054419f740 con 0x7f0544101050 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f054359e640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0544101a10 0x7f054419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f0542d9d640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05440ffe70 0x7f0544197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f054259c640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0544101050 0x7f0544198060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f053800e9f0 tx=0x7f053800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f052bfff640 1 -- 172.21.15.92:0/850202645 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053800ccc0 con 0x7f0544101050 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f054911b640 1 -- 172.21.15.92:0/850202645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054419fa30 con 0x7f0544101050 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f052bfff640 1 -- 172.21.15.92:0/850202645 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0538005020 con 0x7f0544101050 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f052bfff640 1 -- 172.21.15.92:0/850202645 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380049e0 con 0x7f0544101050 2024-03-19T02:02:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.595+0000 7f054911b640 1 -- 172.21.15.92:0/850202645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054419ff80 con 0x7f0544101050 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.597+0000 7f052bfff640 1 -- 172.21.15.92:0/850202645 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f0538005560 con 0x7f0544101050 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.598+0000 7f052bfff640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f052c0a28a0 0x7f052c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.598+0000 7f0542d9d640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f052c0a28a0 0x7f052c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.598+0000 7f052bfff640 1 -- 172.21.15.92:0/850202645 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f05380ceaa0 con 0x7f0544101050 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.598+0000 7f054bde1740 1 Processor -- start 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.598+0000 7f054bde1740 1 -- start start 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f0542d9d640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f052c0a28a0 0x7f052c0a4d60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0534000c00 tx=0x7f0534005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054bde1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf23420520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054bde1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf234255f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054bde1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf234280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054bde1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf23384be0 con 0x55bf23420af0 2024-03-19T02:02:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054bde1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf2332d960 con 0x55bf23420100 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054bde1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf23380c80 con 0x55bf23425c60 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf234255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f0542d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf23420520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054359e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf234280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f0542d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf23420520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:58580/0 (socket says 172.21.15.92:58580) 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f0542d9d640 1 -- 172.21.15.92:0/1068040279 learned_addr learned my addr 172.21.15.92:0/1068040279 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf234255f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49156/0 (socket says 172.21.15.92:49156) 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054259c640 1 -- 172.21.15.92:0/1068040279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 msgr2=0x55bf234280f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054259c640 1 --2- 172.21.15.92:0/1068040279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf234280f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054259c640 1 -- 172.21.15.92:0/1068040279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 msgr2=0x55bf23420520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054259c640 1 --2- 172.21.15.92:0/1068040279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf23420520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054259c640 1 -- 172.21.15.92:0/1068040279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf23324d20 con 0x55bf23420af0 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f054259c640 1 --2- 172.21.15.92:0/1068040279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf234255f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f053809b0f0 tx=0x7f053809b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a25919a28603dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.599+0000 7f05297fa640 1 -- 172.21.15.92:0/1068040279 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380b4070 con 0x55bf23420af0 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f05297fa640 1 -- 172.21.15.92:0/1068040279 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053809f540 con 0x55bf23420af0 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f05297fa640 1 -- 172.21.15.92:0/1068040279 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053801c040 con 0x55bf23420af0 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f054bde1740 1 -- 172.21.15.92:0/1068040279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 msgr2=0x55bf234255f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f054bde1740 1 --2- 172.21.15.92:0/1068040279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf234255f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f053809b0f0 tx=0x7f053809b2d0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f054bde1740 1 -- 172.21.15.92:0/1068040279 shutdown_connections 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f054bde1740 1 --2- 172.21.15.92:0/1068040279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf234280f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f054bde1740 1 --2- 172.21.15.92:0/1068040279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf234255f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f054bde1740 1 --2- 172.21.15.92:0/1068040279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf23420520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f054bde1740 1 -- 172.21.15.92:0/1068040279 >> 172.21.15.92:0/1068040279 conn(0x55bf2341b670 msgr2=0x55bf2341ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f054bde1740 1 -- 172.21.15.92:0/1068040279 shutdown_connections 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.600+0000 7f054bde1740 1 -- 172.21.15.92:0/1068040279 wait complete. 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054bde1740 1 -- start start 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054bde1740 1 Processor -- start 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054bde1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf2342a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054bde1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf2342a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054bde1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf23431690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054bde1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bf23428c10 con 0x55bf23420af0 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054bde1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf23340db0 con 0x55bf23420100 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054bde1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf23428d90 con 0x55bf23425c60 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054bde1740 10 client.0 _ll_register_callbacks cb 0x55bf23345e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf2342a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f0542d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf2342a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf2342a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49158/0 (socket says 172.21.15.92:49158) 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054259c640 1 -- 172.21.15.92:0/1807818152 learned_addr learned my addr 172.21.15.92:0/1807818152 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.601+0000 7f054359e640 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf23431690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f054259c640 1 -- 172.21.15.92:0/1807818152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 msgr2=0x55bf23431690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f054259c640 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf23431690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f054259c640 1 -- 172.21.15.92:0/1807818152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 msgr2=0x55bf2342a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f054259c640 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf2342a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f054259c640 1 -- 172.21.15.92:0/1807818152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf2333fc60 con 0x55bf23420af0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f054259c640 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf2342a650 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f053800c6a0 tx=0x7f053809b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64193484d52aaab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f0526ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f0526ff5640 1 -- 172.21.15.92:0/1807818152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053801c070 con 0x55bf23420af0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f0526ff5640 1 -- 172.21.15.92:0/1807818152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f05380a3900 con 0x55bf23420af0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f054bde1740 1 -- 172.21.15.92:0/1807818152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f053809f540 con 0x55bf23420af0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f0526ff5640 1 -- 172.21.15.92:0/1807818152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05380b4040 con 0x55bf23420af0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f054bde1740 10 client.16086 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f054bde1740 1 -- 172.21.15.92:0/1807818152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf23099210 con 0x55bf23420af0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f0526ff5640 1 -- 172.21.15.92:0/1807818152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053809f540 con 0x55bf23420af0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.602+0000 7f0526ff5640 1 -- 172.21.15.92:0/1807818152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0538097140 con 0x55bf23420af0 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.603+0000 7f054bde1740 10 client.16086 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.603+0000 7f054bde1740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.603+0000 7f054bde1740 20 client.16086 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.603+0000 7f054bde1740 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf23435b80 0x55bf23437f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.603+0000 7f054bde1740 4 client.16086 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.603+0000 7f054bde1740 1 -- 172.21.15.92:0/1807818152 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf23392120 con 0x55bf23435b80 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.603+0000 7f0542d9d640 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf23435b80 0x55bf23437f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.603+0000 7f0542d9d640 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf23435b80 0x55bf23437f70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c7e7356d19defe server_cookie=4c0fa275fca8acba in_seq=0 out_seq=0 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.603+0000 7f0526ff5640 10 client.16086 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f0526ff5640 1 -- 172.21.15.92:0/1807818152 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bf23392120 con 0x55bf23435b80 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f0526ff5640 10 client.16086 handle_command_reply: tid=0 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 client.16086 shutdown 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 -- 172.21.15.92:0/1807818152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 msgr2=0x55bf2342a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf2342a650 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f053800c6a0 tx=0x7f053809b4b0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 -- 172.21.15.92:0/1807818152 shutdown_connections 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf23435b80 0x55bf23437f70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf23431690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf2342a650 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 --2- 172.21.15.92:0/1807818152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf2342a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 -- 172.21.15.92:0/1807818152 >> 172.21.15.92:0/1807818152 conn(0x55bf2341b670 msgr2=0x55bf2341c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f0526ff5640 0 client.16086 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 -- 172.21.15.92:0/1807818152 shutdown_connections 2024-03-19T02:02:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 1 -- 172.21.15.92:0/1807818152 wait complete. 2024-03-19T02:02:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.604+0000 7f054bde1740 20 client.16086 trim_cache size 0 max 16384 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.619+0000 7f054bde1740 1 Processor -- start 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054bde1740 1 -- start start 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054bde1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf2341a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054bde1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf23431650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054bde1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf2341a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054bde1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05441a8cf0 con 0x55bf23420af0 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054bde1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05441aa830 con 0x55bf23420100 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054bde1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f05441a93c0 con 0x55bf23425c60 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf23431650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f0542d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf2341a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf23431650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49172/0 (socket says 172.21.15.92:49172) 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054259c640 1 -- 172.21.15.92:0/1981595475 learned_addr learned my addr 172.21.15.92:0/1981595475 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054359e640 1 --2- 172.21.15.92:0/1981595475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf2341a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f0542d9d640 1 -- 172.21.15.92:0/1981595475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 msgr2=0x55bf2341a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f0542d9d640 1 --2- 172.21.15.92:0/1981595475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf2341a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f0542d9d640 1 -- 172.21.15.92:0/1981595475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 msgr2=0x55bf23431650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f0542d9d640 1 --2- 172.21.15.92:0/1981595475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf23431650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f0542d9d640 1 -- 172.21.15.92:0/1981595475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf23165620 con 0x55bf23420100 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f054259c640 1 --2- 172.21.15.92:0/1981595475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf23431650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.620+0000 7f0542d9d640 1 --2- 172.21.15.92:0/1981595475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf2341a220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0534036040 tx=0x7f053402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a065be5fedd697d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.621+0000 7f0528ff9640 1 -- 172.21.15.92:0/1981595475 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053405a070 con 0x55bf23420100 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.621+0000 7f0528ff9640 1 -- 172.21.15.92:0/1981595475 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0534054d90 con 0x55bf23420100 2024-03-19T02:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.621+0000 7f0528ff9640 1 -- 172.21.15.92:0/1981595475 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534041050 con 0x55bf23420100 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.621+0000 7f054bde1740 1 -- 172.21.15.92:0/1981595475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 msgr2=0x55bf2341a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.621+0000 7f054bde1740 1 --2- 172.21.15.92:0/1981595475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf2341a220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0534036040 tx=0x7f053402bc30 comp rx=0 tx=0).stop 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.621+0000 7f054bde1740 1 -- 172.21.15.92:0/1981595475 shutdown_connections 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.621+0000 7f054bde1740 1 --2- 172.21.15.92:0/1981595475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf2341a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.621+0000 7f054bde1740 1 --2- 172.21.15.92:0/1981595475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420af0 0x55bf23431650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.621+0000 7f054bde1740 1 --2- 172.21.15.92:0/1981595475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420100 0x55bf2341a220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 -- 172.21.15.92:0/1981595475 >> 172.21.15.92:0/1981595475 conn(0x55bf2341b670 msgr2=0x55bf23425030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 -- 172.21.15.92:0/1981595475 shutdown_connections 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 -- 172.21.15.92:0/1981595475 wait complete. 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 -- start start 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 Processor -- start 2024-03-19T02:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420100 0x55bf2342f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420af0 0x55bf2342f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf234308d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05441aa160 con 0x55bf23420100 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05441a9a90 con 0x55bf23420af0 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f05441a6cd0 con 0x55bf23425c60 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054bde1740 10 client.0 _ll_register_callbacks cb 0x7f05380b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f0542d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420100 0x55bf2342f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f0542d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420100 0x55bf2342f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49186/0 (socket says 172.21.15.92:49186) 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f0542d9d640 1 -- 172.21.15.92:0/2272805768 learned_addr learned my addr 172.21.15.92:0/2272805768 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054259c640 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420af0 0x55bf2342f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.622+0000 7f054359e640 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf234308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0542d9d640 1 -- 172.21.15.92:0/2272805768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 msgr2=0x55bf234308d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0542d9d640 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf234308d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0542d9d640 1 -- 172.21.15.92:0/2272805768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420af0 msgr2=0x55bf2342f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0542d9d640 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420af0 0x55bf2342f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0542d9d640 1 -- 172.21.15.92:0/2272805768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf23430ec0 con 0x55bf23420100 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f054259c640 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420af0 0x55bf2342f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0542d9d640 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420100 0x55bf2342f380 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f053402c170 tx=0x7f0534054c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af115149779cdc83 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0526ff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0526ff5640 1 -- 172.21.15.92:0/2272805768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053405a030 con 0x55bf23420100 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0526ff5640 1 -- 172.21.15.92:0/2272805768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0534005040 con 0x55bf23420100 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f054bde1740 1 -- 172.21.15.92:0/2272805768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0534054d90 con 0x55bf23420100 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0526ff5640 1 -- 172.21.15.92:0/2272805768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0534041020 con 0x55bf23420100 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f054bde1740 10 client.16092 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f054bde1740 1 -- 172.21.15.92:0/2272805768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf2349c2c0 con 0x55bf23420100 2024-03-19T02:02:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.623+0000 7f0526ff5640 1 -- 172.21.15.92:0/2272805768 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0534054d90 con 0x55bf23420100 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f0526ff5640 1 -- 172.21.15.92:0/2272805768 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0534004bb0 con 0x55bf23420100 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f054bde1740 10 client.16092 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f054bde1740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f054bde1740 20 client.16092 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f054bde1740 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf2349cd20 0x55bf2349f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f054bde1740 4 client.16092 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f054bde1740 1 -- 172.21.15.92:0/2272805768 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf2338c470 con 0x55bf2349cd20 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f054259c640 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf2349cd20 0x55bf2349f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f0526ff5640 10 client.16092 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.624+0000 7f054259c640 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf2349cd20 0x55bf2349f140 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2724a9ec938dcf50 server_cookie=ee922240bdc8aefe in_seq=0 out_seq=0 2024-03-19T02:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.631+0000 7f0526ff5640 1 -- 172.21.15.92:0/2272805768 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf2338c470 con 0x55bf2349cd20 2024-03-19T02:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.631+0000 7f0526ff5640 10 client.16092 handle_command_reply: tid=0 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.631+0000 7f054bde1740 1 client.16092 shutdown 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 -- 172.21.15.92:0/2272805768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420100 msgr2=0x55bf2342f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420100 0x55bf2342f380 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f053402c170 tx=0x7f0534054c20 comp rx=0 tx=0).stop 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 -- 172.21.15.92:0/2272805768 shutdown_connections 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55bf2349cd20 0x55bf2349f140 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf23425c60 0x55bf234308d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf23420af0 0x55bf2342f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f0526ff5640 0 client.16092 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 --2- 172.21.15.92:0/2272805768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bf23420100 0x55bf2342f380 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 -- 172.21.15.92:0/2272805768 >> 172.21.15.92:0/2272805768 conn(0x55bf2341b670 msgr2=0x55bf23425030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 -- 172.21.15.92:0/2272805768 shutdown_connections 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 1 -- 172.21.15.92:0/2272805768 wait complete. 2024-03-19T02:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054bde1740 20 client.16092 trim_cache size 0 max 16384 2024-03-19T02:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 -- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f052c0a28a0 msgr2=0x7f052c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f052c0a28a0 0x7f052c0a4d60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0534000c00 tx=0x7f0534005fd0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 -- 172.21.15.92:0/850202645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0544101050 msgr2=0x7f0544198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0544101050 0x7f0544198060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f053800e9f0 tx=0x7f053800eeb0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 -- 172.21.15.92:0/850202645 shutdown_connections 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0544101a10 0x7f054419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f052c0a28a0 0x7f052c0a4d60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0544101050 0x7f0544198060 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 --2- 172.21.15.92:0/850202645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05440ffe70 0x7f0544197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 -- 172.21.15.92:0/850202645 >> 172.21.15.92:0/850202645 conn(0x7f05440fb470 msgr2=0x7f05441043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.632+0000 7f054911b640 1 -- 172.21.15.92:0/850202645 shutdown_connections 2024-03-19T02:02:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:19.633+0000 7f054911b640 1 -- 172.21.15.92:0/850202645 wait complete. 2024-03-19T02:02:19.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:19 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:02:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:19 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:02:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:19 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:02:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:20 smithi159 ceph-mon[67110]: pgmap v637: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T02:02:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:02:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:02:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:02:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:20 smithi092 ceph-mon[62574]: pgmap v637: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T02:02:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:02:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:02:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:02:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:20 smithi144 ceph-mon[67199]: pgmap v637: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T02:02:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:02:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:02:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:02:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:22 smithi159 ceph-mon[67110]: pgmap v638: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T02:02:23.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:22 smithi092 ceph-mon[62574]: pgmap v638: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T02:02:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:22 smithi144 ceph-mon[67199]: pgmap v638: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-03-19T02:02:24.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:24 smithi159 ceph-mon[67110]: pgmap v639: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:25.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:24 smithi092 ceph-mon[62574]: pgmap v639: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:25.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:24 smithi144 ceph-mon[67199]: pgmap v639: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:26 smithi159 ceph-mon[67110]: pgmap v640: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:02:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:26 smithi092 ceph-mon[62574]: pgmap v640: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:02:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:26 smithi144 ceph-mon[67199]: pgmap v640: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:02:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:28 smithi159 ceph-mon[67110]: pgmap v641: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:02:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:28 smithi092 ceph-mon[62574]: pgmap v641: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:02:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:28 smithi144 ceph-mon[67199]: pgmap v641: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:02:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:30 smithi159 ceph-mon[67110]: pgmap v642: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T02:02:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:30 smithi092 ceph-mon[62574]: pgmap v642: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T02:02:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:30 smithi144 ceph-mon[67199]: pgmap v642: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-03-19T02:02:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.731+0000 7fdcd6d09640 1 -- 172.21.15.92:0/1676843518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0103420 msgr2=0x7fdcd0109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.731+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/1676843518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0103420 0x7fdcd0109cb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009900 tx=0x7fdcc002bc30 comp rx=0 tx=0).stop 2024-03-19T02:02:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.731+0000 7fdcd6d09640 1 -- 172.21.15.92:0/1676843518 shutdown_connections 2024-03-19T02:02:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.731+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/1676843518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0103420 0x7fdcd0109cb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.731+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/1676843518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcd0102a60 0x7fdcd0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.731+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/1676843518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcd0101870 0x7fdcd0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.731+0000 7fdcd6d09640 1 -- 172.21.15.92:0/1676843518 >> 172.21.15.92:0/1676843518 conn(0x7fdcd00fce00 msgr2=0x7fdcd00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.731+0000 7fdcd6d09640 1 -- 172.21.15.92:0/1676843518 shutdown_connections 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.731+0000 7fdcd6d09640 1 -- 172.21.15.92:0/1676843518 wait complete. 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd6d09640 1 Processor -- start 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd6d09640 1 -- start start 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd6d09640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0101870 0x7fdcd0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd6d09640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcd0102a60 0x7fdcd0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd6d09640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcd0103420 0x7fdcd019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd6d09640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010d540 con 0x7fdcd0101870 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd6d09640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010d3c0 con 0x7fdcd0103420 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd6d09640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010d6c0 con 0x7fdcd0102a60 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd4aa5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0101870 0x7fdcd0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd4aa5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0101870 0x7fdcd0197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37754/0 (socket says 172.21.15.92:37754) 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd4aa5640 1 -- 172.21.15.92:0/2529375734 learned_addr learned my addr 172.21.15.92:0/2529375734 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdccffff640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcd0102a60 0x7fdcd0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd52a6640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcd0103420 0x7fdcd019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd4aa5640 1 -- 172.21.15.92:0/2529375734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcd0102a60 msgr2=0x7fdcd0197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcd0102a60 0x7fdcd0197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd4aa5640 1 -- 172.21.15.92:0/2529375734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcd0103420 msgr2=0x7fdcd019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcd0103420 0x7fdcd019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd4aa5640 1 -- 172.21.15.92:0/2529375734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd019f680 con 0x7fdcd0101870 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd52a6640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcd0103420 0x7fdcd019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0101870 0x7fdcd0197a60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800ebc0 tx=0x7fdcc800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.732+0000 7fdccdffb640 1 -- 172.21.15.92:0/2529375734 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8003b70 con 0x7fdcd0101870 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.733+0000 7fdccdffb640 1 -- 172.21.15.92:0/2529375734 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc8003d10 con 0x7fdcd0101870 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.733+0000 7fdcd6d09640 1 -- 172.21.15.92:0/2529375734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd019f970 con 0x7fdcd0101870 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.733+0000 7fdccdffb640 1 -- 172.21.15.92:0/2529375734 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8019980 con 0x7fdcd0101870 2024-03-19T02:02:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.733+0000 7fdcd6d09640 1 -- 172.21.15.92:0/2529375734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd019ff10 con 0x7fdcd0101870 2024-03-19T02:02:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.735+0000 7fdccdffb640 1 -- 172.21.15.92:0/2529375734 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fdcc8019b20 con 0x7fdcd0101870 2024-03-19T02:02:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.735+0000 7fdccdffb640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdcb80a2920 0x7fdcb80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.735+0000 7fdccffff640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdcb80a2920 0x7fdcb80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.735+0000 7fdccdffb640 1 -- 172.21.15.92:0/2529375734 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdcc80cf260 con 0x7fdcd0101870 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdccffff640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdcb80a2920 0x7fdcb80a4de0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0198f70 tx=0x7fdcc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd99d4740 1 Processor -- start 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd99d4740 1 -- start start 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace97f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97faf0 0x559ace9845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace984c60 0x559ace9870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ace8e3be0 con 0x559ace984c60 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559ace88c960 con 0x559ace97faf0 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559ace8dfc80 con 0x559ace97f100 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd52a6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace984c60 0x559ace9870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.736+0000 7fdcd4aa5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace97f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcd4aa5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace97f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:51488/0 (socket says 172.21.15.92:51488) 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcd4aa5640 1 -- 172.21.15.92:0/474803975 learned_addr learned my addr 172.21.15.92:0/474803975 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdccffff640 1 --2- 172.21.15.92:0/474803975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97faf0 0x559ace9845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcd4aa5640 1 -- 172.21.15.92:0/474803975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97faf0 msgr2=0x559ace9845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/474803975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97faf0 0x559ace9845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcd4aa5640 1 -- 172.21.15.92:0/474803975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace984c60 msgr2=0x559ace9870f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/474803975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace984c60 0x559ace9870f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcd4aa5640 1 -- 172.21.15.92:0/474803975 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ace883d20 con 0x559ace97f100 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/474803975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace97f520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009270 tx=0x7fdcc8021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c1ce367e8edeb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcb6ffd640 1 -- 172.21.15.92:0/474803975 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc809bc10 con 0x559ace97f100 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcb6ffd640 1 -- 172.21.15.92:0/474803975 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc809fd60 con 0x559ace97f100 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.737+0000 7fdcb6ffd640 1 -- 172.21.15.92:0/474803975 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80a4180 con 0x559ace97f100 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 -- 172.21.15.92:0/474803975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 msgr2=0x559ace97f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/474803975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace97f520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009270 tx=0x7fdcc8021000 comp rx=0 tx=0).stop 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 -- 172.21.15.92:0/474803975 shutdown_connections 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/474803975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace984c60 0x559ace9870f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/474803975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97faf0 0x559ace9845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/474803975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace97f520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 -- 172.21.15.92:0/474803975 >> 172.21.15.92:0/474803975 conn(0x559ace97a670 msgr2=0x559ace97aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 -- 172.21.15.92:0/474803975 shutdown_connections 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 -- 172.21.15.92:0/474803975 wait complete. 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 -- start start 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.738+0000 7fdcd99d4740 1 Processor -- start 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97f100 0x559ace9890e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97faf0 0x559ace989620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 0x559ace990660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ace987c10 con 0x559ace97f100 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559ace89fdb0 con 0x559ace984c60 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559ace987d90 con 0x559ace97faf0 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd99d4740 10 client.0 _ll_register_callbacks cb 0x559ace8a4e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd4aa5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97f100 0x559ace9890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdccffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97faf0 0x559ace989620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd4aa5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97f100 0x559ace9890e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37766/0 (socket says 172.21.15.92:37766) 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd4aa5640 1 -- 172.21.15.92:0/3748205688 learned_addr learned my addr 172.21.15.92:0/3748205688 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd52a6640 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 0x559ace990660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdccffff640 1 -- 172.21.15.92:0/3748205688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 msgr2=0x559ace990660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdccffff640 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 0x559ace990660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdccffff640 1 -- 172.21.15.92:0/3748205688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97f100 msgr2=0x559ace9890e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdccffff640 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97f100 0x559ace9890e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdccffff640 1 -- 172.21.15.92:0/3748205688 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ace89ec60 con 0x559ace97faf0 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97f100 0x559ace9890e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdccffff640 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97faf0 0x559ace989620 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x559ace980560 tx=0x7fdcc4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46974741d1326c70 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcaffff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcaffff640 1 -- 172.21.15.92:0/3748205688 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc401fe50 con 0x559ace97faf0 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcaffff640 1 -- 172.21.15.92:0/3748205688 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdcc40047a0 con 0x559ace97faf0 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.739+0000 7fdcd99d4740 1 -- 172.21.15.92:0/3748205688 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc809fd60 con 0x559ace97faf0 2024-03-19T02:02:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcaffff640 1 -- 172.21.15.92:0/3748205688 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdcc40049e0 con 0x559ace97faf0 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcd99d4740 10 client.25990 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcd99d4740 1 -- 172.21.15.92:0/3748205688 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ace5f8210 con 0x559ace97faf0 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcaffff640 1 -- 172.21.15.92:0/3748205688 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc809fd60 con 0x559ace97faf0 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcaffff640 1 -- 172.21.15.92:0/3748205688 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdcc4026070 con 0x559ace97faf0 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcd99d4740 10 client.25990 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcd99d4740 10 client.25990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcd99d4740 20 client.25990 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x559ace994f70 0x559ace9973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcd99d4740 4 client.25990 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcd99d4740 1 -- 172.21.15.92:0/3748205688 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ace8f1120 con 0x559ace994f70 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.740+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x559ace994f70 0x559ace9973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.741+0000 7fdcaffff640 10 client.25990 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.741+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x559ace994f70 0x559ace9973e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec99fc8130bcc51e server_cookie=5d34061e902d082a in_seq=0 out_seq=0 2024-03-19T02:02:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcaffff640 1 -- 172.21.15.92:0/3748205688 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559ace8f1120 con 0x559ace994f70 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcaffff640 10 client.25990 handle_command_reply: tid=0 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 client.25990 shutdown 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 -- 172.21.15.92:0/3748205688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97faf0 msgr2=0x559ace989620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97faf0 0x559ace989620 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x559ace980560 tx=0x7fdcc4004730 comp rx=0 tx=0).stop 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 -- 172.21.15.92:0/3748205688 shutdown_connections 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x559ace994f70 0x559ace9973e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 0x559ace990660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcaffff640 0 client.25990 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97faf0 0x559ace989620 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/3748205688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97f100 0x559ace9890e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 -- 172.21.15.92:0/3748205688 >> 172.21.15.92:0/3748205688 conn(0x559ace97a670 msgr2=0x559ace97b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 -- 172.21.15.92:0/3748205688 shutdown_connections 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 1 -- 172.21.15.92:0/3748205688 wait complete. 2024-03-19T02:02:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.742+0000 7fdcd99d4740 20 client.25990 trim_cache size 0 max 16384 2024-03-19T02:02:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.756+0000 7fdcd99d4740 1 Processor -- start 2024-03-19T02:02:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.756+0000 7fdcd99d4740 1 -- start start 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.756+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97f100 0x559ace97dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.756+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace990620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.756+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace984c60 0x559ace99bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.756+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01a8cb0 con 0x559ace97faf0 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01aa7f0 con 0x559ace97f100 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01a9380 con 0x559ace984c60 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdccffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace990620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdccffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace990620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37774/0 (socket says 172.21.15.92:37774) 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdccffff640 1 -- 172.21.15.92:0/2722322546 learned_addr learned my addr 172.21.15.92:0/2722322546 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdcd52a6640 1 --2- 172.21.15.92:0/2722322546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace984c60 0x559ace99bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/2722322546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97f100 0x559ace97dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdccffff640 1 -- 172.21.15.92:0/2722322546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace984c60 msgr2=0x559ace99bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdccffff640 1 --2- 172.21.15.92:0/2722322546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace984c60 0x559ace99bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdccffff640 1 -- 172.21.15.92:0/2722322546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97f100 msgr2=0x559ace97dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdccffff640 1 --2- 172.21.15.92:0/2722322546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97f100 0x559ace97dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdccffff640 1 -- 172.21.15.92:0/2722322546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ace6c4620 con 0x559ace97faf0 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdccffff640 1 --2- 172.21.15.92:0/2722322546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace990620 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40315b0 tx=0x7fdcc4003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225695c03ba93005 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdcb67fc640 1 -- 172.21.15.92:0/2722322546 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40035f0 con 0x559ace97faf0 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdcb67fc640 1 -- 172.21.15.92:0/2722322546 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc4036070 con 0x559ace97faf0 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.757+0000 7fdcb67fc640 1 -- 172.21.15.92:0/2722322546 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40279d0 con 0x559ace97faf0 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2722322546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 msgr2=0x559ace990620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2722322546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace990620 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40315b0 tx=0x7fdcc4003fe0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2722322546 shutdown_connections 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2722322546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace984c60 0x559ace99bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2722322546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace990620 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2722322546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace97f100 0x559ace97dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2722322546 >> 172.21.15.92:0/2722322546 conn(0x559ace97a670 msgr2=0x559ace97d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2722322546 shutdown_connections 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2722322546 wait complete. 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 -- start start 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.758+0000 7fdcd99d4740 1 Processor -- start 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace98e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace98eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd99d4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 0x559ace98fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01aa120 con 0x559ace97faf0 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01a9a50 con 0x559ace984c60 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd99d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01a6c90 con 0x559ace97f100 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd99d4740 10 client.0 _ll_register_callbacks cb 0x7fdcc4004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdccffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace98eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd52a6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 0x559ace98fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdccffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace98eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37788/0 (socket says 172.21.15.92:37788) 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd4aa5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace98e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdccffff640 1 -- 172.21.15.92:0/2100072633 learned_addr learned my addr 172.21.15.92:0/2100072633 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdccffff640 1 -- 172.21.15.92:0/2100072633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 msgr2=0x559ace98e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdccffff640 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace98e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdccffff640 1 -- 172.21.15.92:0/2100072633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 msgr2=0x559ace98fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdccffff640 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 0x559ace98fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdccffff640 1 -- 172.21.15.92:0/2100072633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ace990030 con 0x559ace97faf0 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace98e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcd52a6640 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 0x559ace98fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdccffff640 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace98eaf0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40035f0 tx=0x7fdcc4003620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2aee565a628d7f server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcaffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.759+0000 7fdcaffff640 1 -- 172.21.15.92:0/2100072633 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4047060 con 0x559ace97faf0 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2100072633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc4036070 con 0x559ace97faf0 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcaffff640 1 -- 172.21.15.92:0/2100072633 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdcc4031930 con 0x559ace97faf0 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcaffff640 1 -- 172.21.15.92:0/2100072633 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdcc40047d0 con 0x559ace97faf0 2024-03-19T02:02:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd99d4740 10 client.16110 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2100072633 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ace9fba90 con 0x559ace97faf0 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcaffff640 1 -- 172.21.15.92:0/2100072633 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc4036070 con 0x559ace97faf0 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcaffff640 1 -- 172.21.15.92:0/2100072633 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdcc4026070 con 0x559ace97faf0 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd99d4740 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd99d4740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd99d4740 20 client.16110 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x559ace9fc3b0 0x559ace9fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd99d4740 4 client.16110 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2100072633 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ace8eb470 con 0x559ace9fc3b0 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.760+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x559ace9fc3b0 0x559ace9fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.761+0000 7fdcd4aa5640 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x559ace9fc3b0 0x559ace9fe7d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3f24e39a23d222 server_cookie=3d3b20cd74e16602 in_seq=0 out_seq=0 2024-03-19T02:02:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.761+0000 7fdcaffff640 10 client.16110 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcaffff640 1 -- 172.21.15.92:0/2100072633 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ace8eb470 con 0x559ace9fc3b0 2024-03-19T02:02:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcaffff640 10 client.16110 handle_command_reply: tid=0 2024-03-19T02:02:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 client.16110 shutdown 2024-03-19T02:02:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2100072633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 msgr2=0x559ace98eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace98eaf0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40035f0 tx=0x7fdcc4003620 comp rx=0 tx=0).stop 2024-03-19T02:02:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2100072633 shutdown_connections 2024-03-19T02:02:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x559ace9fc3b0 0x559ace9fe7d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ace984c60 0x559ace98fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcaffff640 0 client.16110 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ace97faf0 0x559ace98eaf0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 --2- 172.21.15.92:0/2100072633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ace97f100 0x559ace98e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2100072633 >> 172.21.15.92:0/2100072633 conn(0x559ace97a670 msgr2=0x559ace97e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2100072633 shutdown_connections 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.767+0000 7fdcd99d4740 1 -- 172.21.15.92:0/2100072633 wait complete. 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd99d4740 20 client.16110 trim_cache size 0 max 16384 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:02:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:02:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:02:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 -- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdcb80a2920 msgr2=0x7fdcb80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdcb80a2920 0x7fdcb80a4de0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0198f70 tx=0x7fdcc4005f90 comp rx=0 tx=0).stop 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 -- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0101870 msgr2=0x7fdcd0197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0101870 0x7fdcd0197a60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800ebc0 tx=0x7fdcc800c6a0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 -- 172.21.15.92:0/2529375734 shutdown_connections 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcd0103420 0x7fdcd019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdcb80a2920 0x7fdcb80a4de0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcd0102a60 0x7fdcd0197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 --2- 172.21.15.92:0/2529375734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcd0101870 0x7fdcd0197a60 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 -- 172.21.15.92:0/2529375734 >> 172.21.15.92:0/2529375734 conn(0x7fdcd00fce00 msgr2=0x7fdcd0105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 -- 172.21.15.92:0/2529375734 shutdown_connections 2024-03-19T02:02:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:32.768+0000 7fdcd6d09640 1 -- 172.21.15.92:0/2529375734 wait complete. 2024-03-19T02:02:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:32 smithi159 ceph-mon[67110]: pgmap v643: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:02:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:32 smithi092 ceph-mon[62574]: pgmap v643: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:02:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:32 smithi144 ceph-mon[67199]: pgmap v643: 97 pgs: 97 active+clean; 3.1 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:02:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:34 smithi159 ceph-mon[67110]: pgmap v644: 97 pgs: 97 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:02:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:34 smithi092 ceph-mon[62574]: pgmap v644: 97 pgs: 97 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:02:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:34 smithi144 ceph-mon[67199]: pgmap v644: 97 pgs: 97 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:02:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:36 smithi159 ceph-mon[67110]: pgmap v645: 97 pgs: 97 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T02:02:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:36 smithi092 ceph-mon[62574]: pgmap v645: 97 pgs: 97 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T02:02:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:36 smithi144 ceph-mon[67199]: pgmap v645: 97 pgs: 97 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T02:02:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:38 smithi159 ceph-mon[67110]: pgmap v646: 97 pgs: 97 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:02:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:38 smithi092 ceph-mon[62574]: pgmap v646: 97 pgs: 97 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:02:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:38 smithi144 ceph-mon[67199]: pgmap v646: 97 pgs: 97 active+clean; 3.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:02:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:40 smithi159 ceph-mon[67110]: pgmap v647: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:40 smithi092 ceph-mon[62574]: pgmap v647: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:40 smithi144 ceph-mon[67199]: pgmap v647: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:43.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:42 smithi092 ceph-mon[62574]: pgmap v648: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:02:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:42 smithi144 ceph-mon[67199]: pgmap v648: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:02:43.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:42 smithi159 ceph-mon[67110]: pgmap v648: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:02:45.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:44 smithi092 ceph-mon[62574]: pgmap v649: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:45.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:44 smithi144 ceph-mon[67199]: pgmap v649: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:45.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:44 smithi159 ceph-mon[67110]: pgmap v649: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:02:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 -- 172.21.15.92:0/277561413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e81022f0 msgr2=0x7f74e8108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 --2- 172.21.15.92:0/277561413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e81022f0 0x7f74e8108b80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009900 tx=0x7f74e002bc30 comp rx=0 tx=0).stop 2024-03-19T02:02:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 -- 172.21.15.92:0/277561413 shutdown_connections 2024-03-19T02:02:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 --2- 172.21.15.92:0/277561413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e81022f0 0x7f74e8108b80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 --2- 172.21.15.92:0/277561413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e8101930 0x7f74e8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 --2- 172.21.15.92:0/277561413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74e8100740 0x7f74e8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 -- 172.21.15.92:0/277561413 >> 172.21.15.92:0/277561413 conn(0x7f74e80fbcb0 msgr2=0x7f74e80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 -- 172.21.15.92:0/277561413 shutdown_connections 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 -- 172.21.15.92:0/277561413 wait complete. 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 Processor -- start 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 -- start start 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74e8100740 0x7f74e810b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8101930 0x7f74e8109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e81022f0 0x7f74e8109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74e810d4e0 con 0x7f74e8101930 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74e810d360 con 0x7f74e81022f0 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.859+0000 7f74eecbd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f74e810d660 con 0x7f74e8100740 2024-03-19T02:02:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8101930 0x7f74e8109700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74eca59640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74e8100740 0x7f74e810b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8101930 0x7f74e8109700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51692/0 (socket says 172.21.15.92:51692) 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74dffff640 1 -- 172.21.15.92:0/3333736367 learned_addr learned my addr 172.21.15.92:0/3333736367 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74ed25a640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e81022f0 0x7f74e8109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74dffff640 1 -- 172.21.15.92:0/3333736367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74e8100740 msgr2=0x7f74e810b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74dffff640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74e8100740 0x7f74e810b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74dffff640 1 -- 172.21.15.92:0/3333736367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e81022f0 msgr2=0x7f74e8109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74dffff640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e81022f0 0x7f74e8109c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74dffff640 1 -- 172.21.15.92:0/3333736367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e810a4d0 con 0x7f74e8101930 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74eca59640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74e8100740 0x7f74e810b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74dffff640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8101930 0x7f74e8109700 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e9f0 tx=0x7f74d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74ddffb640 1 -- 172.21.15.92:0/3333736367 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d800cd80 con 0x7f74e8101930 2024-03-19T02:02:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74eecbd640 1 -- 172.21.15.92:0/3333736367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e81a1ba0 con 0x7f74e8101930 2024-03-19T02:02:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74ddffb640 1 -- 172.21.15.92:0/3333736367 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74d8010910 con 0x7f74e8101930 2024-03-19T02:02:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74ddffb640 1 -- 172.21.15.92:0/3333736367 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d8018950 con 0x7f74e8101930 2024-03-19T02:02:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.860+0000 7f74eecbd640 1 -- 172.21.15.92:0/3333736367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e81a2140 con 0x7f74e8101930 2024-03-19T02:02:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74ddffb640 1 -- 172.21.15.92:0/3333736367 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f74d8010ac0 con 0x7f74e8101930 2024-03-19T02:02:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74f1983740 1 Processor -- start 2024-03-19T02:02:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74f1983740 1 -- start start 2024-03-19T02:02:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74f1983740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74f1983740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f80135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74f1983740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f80160f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74f1983740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5604f7f72be0 con 0x5604f800e100 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74f1983740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604f7f1b960 con 0x5604f800eaf0 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74f1983740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604f7f6ec80 con 0x5604f8013c60 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74ed25a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74ed25a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51700/0 (socket says 172.21.15.92:51700) 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74ed25a640 1 -- 172.21.15.92:0/1674935640 learned_addr learned my addr 172.21.15.92:0/1674935640 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74eca59640 1 --2- 172.21.15.92:0/1674935640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f80135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74dffff640 1 --2- 172.21.15.92:0/1674935640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f80160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.863+0000 7f74ddffb640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f74d00a28a0 0x7f74d00a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ed25a640 1 -- 172.21.15.92:0/1674935640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 msgr2=0x5604f80160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ed25a640 1 --2- 172.21.15.92:0/1674935640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f80160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ed25a640 1 -- 172.21.15.92:0/1674935640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 msgr2=0x5604f80135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ed25a640 1 --2- 172.21.15.92:0/1674935640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f80135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ed25a640 1 -- 172.21.15.92:0/1674935640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f7f12d20 con 0x5604f800e100 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ed25a640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f74d00a28a0 0x7f74d00a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ddffb640 1 -- 172.21.15.92:0/3333736367 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f74d80cedd0 con 0x7f74e8101930 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ed25a640 1 --2- 172.21.15.92:0/1674935640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800e520 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f74e0006010 tx=0x7f74e00095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0af51b5c98189a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ceffd640 1 -- 172.21.15.92:0/1674935640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e004f070 con 0x5604f800e100 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ceffd640 1 -- 172.21.15.92:0/1674935640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e00053d0 con 0x5604f800e100 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.864+0000 7f74ceffd640 1 -- 172.21.15.92:0/1674935640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0049a30 con 0x5604f800e100 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 -- 172.21.15.92:0/1674935640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 msgr2=0x5604f800e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74ed25a640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f74d00a28a0 0x7f74d00a4d60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f74e002c710 tx=0x7f74e0049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 --2- 172.21.15.92:0/1674935640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800e520 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f74e0006010 tx=0x7f74e00095e0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 -- 172.21.15.92:0/1674935640 shutdown_connections 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 --2- 172.21.15.92:0/1674935640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f80160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 --2- 172.21.15.92:0/1674935640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f80135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 --2- 172.21.15.92:0/1674935640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800e520 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 -- 172.21.15.92:0/1674935640 >> 172.21.15.92:0/1674935640 conn(0x5604f8009670 msgr2=0x5604f8009a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 -- 172.21.15.92:0/1674935640 shutdown_connections 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 -- 172.21.15.92:0/1674935640 wait complete. 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 -- start start 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 Processor -- start 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f8018110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f800eaf0 0x5604f8018650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.865+0000 7f74f1983740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f8013c60 0x5604f801f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74f1983740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5604f8016c10 con 0x5604f800e100 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74f1983740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604f7f2edb0 con 0x5604f8013c60 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74f1983740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604f8016d90 con 0x5604f800eaf0 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74f1983740 10 client.0 _ll_register_callbacks cb 0x5604f7f33e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74eca59640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f8018110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74eca59640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f8018110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51706/0 (socket says 172.21.15.92:51706) 2024-03-19T02:02:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74eca59640 1 -- 172.21.15.92:0/2352085151 learned_addr learned my addr 172.21.15.92:0/2352085151 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74dffff640 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f800eaf0 0x5604f8018650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74ed25a640 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f8013c60 0x5604f801f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74eca59640 1 -- 172.21.15.92:0/2352085151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f800eaf0 msgr2=0x5604f8018650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74eca59640 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f800eaf0 0x5604f8018650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74eca59640 1 -- 172.21.15.92:0/2352085151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f8013c60 msgr2=0x5604f801f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74eca59640 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f8013c60 0x5604f801f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74eca59640 1 -- 172.21.15.92:0/2352085151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f7f2dc60 con 0x5604f800e100 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74eca59640 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f8018110 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f74e4005fd0 tx=0x7f74e4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bc94d976aeb69a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74c7fff640 1 -- 172.21.15.92:0/2352085151 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4004b20 con 0x5604f800e100 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74c7fff640 1 -- 172.21.15.92:0/2352085151 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f74e4004cc0 con 0x5604f800e100 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74f1983740 1 -- 172.21.15.92:0/2352085151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e00053d0 con 0x5604f800e100 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74c7fff640 1 -- 172.21.15.92:0/2352085151 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74e401c470 con 0x5604f800e100 2024-03-19T02:02:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74f1983740 10 client.16122 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.866+0000 7f74f1983740 1 -- 172.21.15.92:0/2352085151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5604f7c87210 con 0x5604f800e100 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74c7fff640 1 -- 172.21.15.92:0/2352085151 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e00053d0 con 0x5604f800e100 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74c7fff640 1 -- 172.21.15.92:0/2352085151 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f74e4010800 con 0x5604f800e100 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74f1983740 10 client.16122 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74f1983740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74f1983740 20 client.16122 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74f1983740 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5604f8023fd0 0x5604f8026440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74f1983740 4 client.16122 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74f1983740 1 -- 172.21.15.92:0/2352085151 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604f7f80120 con 0x5604f8023fd0 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74dffff640 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5604f8023fd0 0x5604f8026440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74c7fff640 10 client.16122 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.867+0000 7f74dffff640 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5604f8023fd0 0x5604f8026440 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1c6c02377efa42 server_cookie=f30b42d68478e0f1 in_seq=0 out_seq=0 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74c7fff640 1 -- 172.21.15.92:0/2352085151 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5604f7f80120 con 0x5604f8023fd0 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74c7fff640 10 client.16122 handle_command_reply: tid=0 2024-03-19T02:02:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 client.16122 shutdown 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 -- 172.21.15.92:0/2352085151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 msgr2=0x5604f8018110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f8018110 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f74e4005fd0 tx=0x7f74e4012820 comp rx=0 tx=0).stop 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 -- 172.21.15.92:0/2352085151 shutdown_connections 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5604f8023fd0 0x5604f8026440 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f8013c60 0x5604f801f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f800eaf0 0x5604f8018650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74c7fff640 0 client.16122 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 --2- 172.21.15.92:0/2352085151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f8018110 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 -- 172.21.15.92:0/2352085151 >> 172.21.15.92:0/2352085151 conn(0x5604f8009670 msgr2=0x5604f800a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 -- 172.21.15.92:0/2352085151 shutdown_connections 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 1 -- 172.21.15.92:0/2352085151 wait complete. 2024-03-19T02:02:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.868+0000 7f74f1983740 20 client.16122 trim_cache size 0 max 16384 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74f1983740 1 Processor -- start 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74f1983740 1 -- start start 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74f1983740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74f1983740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f801f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74f1983740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f8017730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74f1983740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74e81aae10 con 0x5604f800e100 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74f1983740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74e81ac950 con 0x5604f800eaf0 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74f1983740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f74e81ab4e0 con 0x5604f8013c60 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74eca59640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74eca59640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51712/0 (socket says 172.21.15.92:51712) 2024-03-19T02:02:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74eca59640 1 -- 172.21.15.92:0/85576871 learned_addr learned my addr 172.21.15.92:0/85576871 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74ed25a640 1 --2- 172.21.15.92:0/85576871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f8017730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74dffff640 1 --2- 172.21.15.92:0/85576871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f801f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74eca59640 1 -- 172.21.15.92:0/85576871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 msgr2=0x5604f8017730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74eca59640 1 --2- 172.21.15.92:0/85576871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f8017730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74eca59640 1 -- 172.21.15.92:0/85576871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 msgr2=0x5604f801f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74eca59640 1 --2- 172.21.15.92:0/85576871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f801f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.883+0000 7f74eca59640 1 -- 172.21.15.92:0/85576871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f7d53620 con 0x5604f800e100 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74eca59640 1 --2- 172.21.15.92:0/85576871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800cfe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f74e4000f80 tx=0x7f74e40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865cefa6e581cefe server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74ce7fc640 1 -- 172.21.15.92:0/85576871 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4003970 con 0x5604f800e100 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74ce7fc640 1 -- 172.21.15.92:0/85576871 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e4010430 con 0x5604f800e100 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74ce7fc640 1 -- 172.21.15.92:0/85576871 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e401c870 con 0x5604f800e100 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74f1983740 1 -- 172.21.15.92:0/85576871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 msgr2=0x5604f800cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74f1983740 1 --2- 172.21.15.92:0/85576871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800cfe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f74e4000f80 tx=0x7f74e40021d0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74f1983740 1 -- 172.21.15.92:0/85576871 shutdown_connections 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74f1983740 1 --2- 172.21.15.92:0/85576871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f8017730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74f1983740 1 --2- 172.21.15.92:0/85576871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f801f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74f1983740 1 --2- 172.21.15.92:0/85576871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f800cfe0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.884+0000 7f74f1983740 1 -- 172.21.15.92:0/85576871 >> 172.21.15.92:0/85576871 conn(0x5604f8009670 msgr2=0x5604f800c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 -- 172.21.15.92:0/85576871 shutdown_connections 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 -- 172.21.15.92:0/85576871 wait complete. 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 -- start start 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 Processor -- start 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f801cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f801d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f801e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74e81ac280 con 0x5604f800e100 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74e81abbb0 con 0x5604f800eaf0 2024-03-19T02:02:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f74e81a8df0 con 0x5604f8013c60 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74f1983740 10 client.0 _ll_register_callbacks cb 0x7f74e400f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74eca59640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f801cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74eca59640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f801cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51726/0 (socket says 172.21.15.92:51726) 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74eca59640 1 -- 172.21.15.92:0/3068308486 learned_addr learned my addr 172.21.15.92:0/3068308486 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74ed25a640 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f801e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.885+0000 7f74dffff640 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f801d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74eca59640 1 -- 172.21.15.92:0/3068308486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 msgr2=0x5604f801e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74eca59640 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f801e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74eca59640 1 -- 172.21.15.92:0/3068308486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 msgr2=0x5604f801d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74eca59640 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f801d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74eca59640 1 -- 172.21.15.92:0/3068308486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f801e6f0 con 0x5604f800e100 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74eca59640 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f801cc00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f74e4005fd0 tx=0x7f74e400f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeeba26fa43ec358 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:02:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74c7fff640 1 -- 172.21.15.92:0/3068308486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4010a10 con 0x5604f800e100 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74c7fff640 1 -- 172.21.15.92:0/3068308486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f74e4003b80 con 0x5604f800e100 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74f1983740 1 -- 172.21.15.92:0/3068308486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e4010430 con 0x5604f800e100 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74c7fff640 1 -- 172.21.15.92:0/3068308486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74e4003d80 con 0x5604f800e100 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74f1983740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74f1983740 1 -- 172.21.15.92:0/3068308486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5604f801fa00 con 0x5604f800e100 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74c7fff640 1 -- 172.21.15.92:0/3068308486 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e4010430 con 0x5604f800e100 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74c7fff640 1 -- 172.21.15.92:0/3068308486 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f74e4037070 con 0x5604f800e100 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74f1983740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74f1983740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.886+0000 7f74f1983740 20 client.16128 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.887+0000 7f74f1983740 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5604f808a9f0 0x5604f808ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.887+0000 7f74f1983740 4 client.16128 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.887+0000 7f74f1983740 1 -- 172.21.15.92:0/3068308486 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604f7f7a470 con 0x5604f808a9f0 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.887+0000 7f74dffff640 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5604f808a9f0 0x5604f808ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.887+0000 7f74dffff640 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5604f808a9f0 0x5604f808ce10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be85a8e5f813551a server_cookie=9860b0ea867d8b21 in_seq=0 out_seq=0 2024-03-19T02:02:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.887+0000 7f74c7fff640 10 client.16128 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74c7fff640 1 -- 172.21.15.92:0/3068308486 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604f7f7a470 con 0x5604f808a9f0 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74c7fff640 10 client.16128 handle_command_reply: tid=0 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 client.16128 shutdown 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 -- 172.21.15.92:0/3068308486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 msgr2=0x5604f801cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f801cc00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f74e4005fd0 tx=0x7f74e400f9a0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 -- 172.21.15.92:0/3068308486 shutdown_connections 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5604f808a9f0 0x5604f808ce10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f8013c60 0x5604f801e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604f800eaf0 0x5604f801d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74c7fff640 0 client.16128 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 --2- 172.21.15.92:0/3068308486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604f800e100 0x5604f801cc00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 -- 172.21.15.92:0/3068308486 >> 172.21.15.92:0/3068308486 conn(0x5604f8009670 msgr2=0x5604f800d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 -- 172.21.15.92:0/3068308486 shutdown_connections 2024-03-19T02:02:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 1 -- 172.21.15.92:0/3068308486 wait complete. 2024-03-19T02:02:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.894+0000 7f74f1983740 20 client.16128 trim_cache size 0 max 16384 2024-03-19T02:02:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:02:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:02:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:02:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 -- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f74d00a28a0 msgr2=0x7f74d00a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f74d00a28a0 0x7f74d00a4d60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f74e002c710 tx=0x7f74e0049000 comp rx=0 tx=0).stop 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 -- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8101930 msgr2=0x7f74e8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8101930 0x7f74e8109700 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e9f0 tx=0x7f74d800eeb0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 -- 172.21.15.92:0/3333736367 shutdown_connections 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74e81022f0 0x7f74e8109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74e8101930 0x7f74e8109700 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f74d00a28a0 0x7f74d00a4d60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 --2- 172.21.15.92:0/3333736367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74e8100740 0x7f74e810b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 -- 172.21.15.92:0/3333736367 >> 172.21.15.92:0/3333736367 conn(0x7f74e80fbcb0 msgr2=0x7f74e8104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 -- 172.21.15.92:0/3333736367 shutdown_connections 2024-03-19T02:02:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:45.895+0000 7f74eecbd640 1 -- 172.21.15.92:0/3333736367 wait complete. 2024-03-19T02:02:47.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:46 smithi092 ceph-mon[62574]: pgmap v650: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-03-19T02:02:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:46 smithi144 ceph-mon[67199]: pgmap v650: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-03-19T02:02:47.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:46 smithi159 ceph-mon[67110]: pgmap v650: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-03-19T02:02:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:48 smithi092 ceph-mon[62574]: pgmap v651: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-03-19T02:02:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:48 smithi144 ceph-mon[67199]: pgmap v651: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-03-19T02:02:49.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:48 smithi159 ceph-mon[67110]: pgmap v651: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-03-19T02:02:51.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:50 smithi092 ceph-mon[62574]: pgmap v652: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 KiB/s wr, 19 op/s 2024-03-19T02:02:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:50 smithi144 ceph-mon[67199]: pgmap v652: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 KiB/s wr, 19 op/s 2024-03-19T02:02:51.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:50 smithi159 ceph-mon[67110]: pgmap v652: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 KiB/s wr, 19 op/s 2024-03-19T02:02:53.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:52 smithi092 ceph-mon[62574]: pgmap v653: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-19T02:02:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:52 smithi144 ceph-mon[67199]: pgmap v653: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-19T02:02:53.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:52 smithi159 ceph-mon[67110]: pgmap v653: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-03-19T02:02:55.007 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:54 smithi159 ceph-mon[67110]: pgmap v654: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:02:55.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:54 smithi092 ceph-mon[62574]: pgmap v654: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:02:55.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:54 smithi144 ceph-mon[67199]: pgmap v654: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:02:57.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:56 smithi092 ceph-mon[62574]: pgmap v655: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:02:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:56 smithi144 ceph-mon[67199]: pgmap v655: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:02:57.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:56 smithi159 ceph-mon[67110]: pgmap v655: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-03-19T02:02:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.981+0000 7fe8e6c19640 1 -- 172.21.15.92:0/2416188434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 msgr2=0x7fe8e0108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.981+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/2416188434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 0x7fe8e0108b80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009900 tx=0x7fe8d002bc30 comp rx=0 tx=0).stop 2024-03-19T02:02:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.981+0000 7fe8e6c19640 1 -- 172.21.15.92:0/2416188434 shutdown_connections 2024-03-19T02:02:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.981+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/2416188434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 0x7fe8e0108b80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.981+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/2416188434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e0101930 0x7fe8e0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.981+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/2416188434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e0100740 0x7fe8e0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.981+0000 7fe8e6c19640 1 -- 172.21.15.92:0/2416188434 >> 172.21.15.92:0/2416188434 conn(0x7fe8e00fbcb0 msgr2=0x7fe8e00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 -- 172.21.15.92:0/2416188434 shutdown_connections 2024-03-19T02:02:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 -- 172.21.15.92:0/2416188434 wait complete. 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 Processor -- start 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 -- start start 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e0100740 0x7fe8e0069a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e0101930 0x7fe8e019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 0x7fe8e019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e010c3c0 con 0x7fe8e01022f0 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e010c240 con 0x7fe8e0100740 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e6c19640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e010c540 con 0x7fe8e0101930 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e51b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 0x7fe8e019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e51b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 0x7fe8e019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41462/0 (socket says 172.21.15.92:41462) 2024-03-19T02:02:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8dffff640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e0101930 0x7fe8e019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e51b6640 1 -- 172.21.15.92:0/1289412649 learned_addr learned my addr 172.21.15.92:0/1289412649 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.982+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e0100740 0x7fe8e0069a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8e51b6640 1 -- 172.21.15.92:0/1289412649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e0101930 msgr2=0x7fe8e019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8e51b6640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e0101930 0x7fe8e019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8e51b6640 1 -- 172.21.15.92:0/1289412649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e0100740 msgr2=0x7fe8e0069a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8e51b6640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e0100740 0x7fe8e0069a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8e51b6640 1 -- 172.21.15.92:0/1289412649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e019f6a0 con 0x7fe8e01022f0 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e0100740 0x7fe8e0069a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8e51b6640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 0x7fe8e019ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0005ab0 tx=0x7fe8d002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8ddffb640 1 -- 172.21.15.92:0/1289412649 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0041070 con 0x7fe8e01022f0 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8e6c19640 1 -- 172.21.15.92:0/1289412649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e019f930 con 0x7fe8e01022f0 2024-03-19T02:02:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8ddffb640 1 -- 172.21.15.92:0/1289412649 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d0004450 con 0x7fe8e01022f0 2024-03-19T02:02:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8ddffb640 1 -- 172.21.15.92:0/1289412649 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0003e60 con 0x7fe8e01022f0 2024-03-19T02:02:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.983+0000 7fe8e6c19640 1 -- 172.21.15.92:0/1289412649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e019fe70 con 0x7fe8e01022f0 2024-03-19T02:02:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.986+0000 7fe8ddffb640 1 -- 172.21.15.92:0/1289412649 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fe8d0044070 con 0x7fe8e01022f0 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.986+0000 7fe8ddffb640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe8cc0a28a0 0x7fe8cc0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.986+0000 7fe8e9807740 1 Processor -- start 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e9807740 1 -- start start 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8ddffb640 1 -- 172.21.15.92:0/1289412649 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe8d00f0670 con 0x7fe8e01022f0 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe8cc0a28a0 0x7fe8cc0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 0x5578416b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b0af0 0x5578416b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b5c60 0x5578416b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e9807740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557841614be0 con 0x5578416b5c60 2024-03-19T02:02:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e9807740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5578415bd960 con 0x5578416b0af0 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e9807740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557841610c80 con 0x5578416b0100 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e51b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b5c60 0x5578416b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e51b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b5c60 0x5578416b80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41478/0 (socket says 172.21.15.92:41478) 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b0af0 0x5578416b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e51b6640 1 -- 172.21.15.92:0/3695954692 learned_addr learned my addr 172.21.15.92:0/3695954692 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/3695954692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 0x5578416b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8dffff640 1 -- 172.21.15.92:0/3695954692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 msgr2=0x5578416b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8dffff640 1 --2- 172.21.15.92:0/3695954692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 0x5578416b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8dffff640 1 -- 172.21.15.92:0/3695954692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b5c60 msgr2=0x5578416b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8dffff640 1 --2- 172.21.15.92:0/3695954692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b5c60 0x5578416b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8dffff640 1 -- 172.21.15.92:0/3695954692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578415b4d20 con 0x5578416b0af0 2024-03-19T02:02:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e51b6640 1 --2- 172.21.15.92:0/3695954692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b5c60 0x5578416b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe8cc0a28a0 0x7fe8cc0a4d60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0101790 tx=0x7fe8d800ff30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.987+0000 7fe8dffff640 1 --2- 172.21.15.92:0/3695954692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b0af0 0x5578416b55f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009bd0 tx=0x7fe8c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d46cb21706aa4ee server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8d6ffd640 1 -- 172.21.15.92:0/3695954692 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8004050 con 0x5578416b0af0 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8d6ffd640 1 -- 172.21.15.92:0/3695954692 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c8005500 con 0x5578416b0af0 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8d6ffd640 1 -- 172.21.15.92:0/3695954692 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8004da0 con 0x5578416b0af0 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 -- 172.21.15.92:0/3695954692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b0af0 msgr2=0x5578416b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3695954692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b0af0 0x5578416b55f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009bd0 tx=0x7fe8c8011810 comp rx=0 tx=0).stop 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 -- 172.21.15.92:0/3695954692 shutdown_connections 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3695954692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b5c60 0x5578416b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3695954692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b0af0 0x5578416b55f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3695954692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 0x5578416b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 -- 172.21.15.92:0/3695954692 >> 172.21.15.92:0/3695954692 conn(0x5578416ab670 msgr2=0x5578416aba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 -- 172.21.15.92:0/3695954692 shutdown_connections 2024-03-19T02:02:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 -- 172.21.15.92:0/3695954692 wait complete. 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.988+0000 7fe8e9807740 1 -- start start 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e9807740 1 Processor -- start 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 0x5578416ba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416c1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e9807740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5578416b8c10 con 0x5578416b0100 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e9807740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5578415d0db0 con 0x5578416b5c60 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e9807740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5578416b8d90 con 0x5578416b0af0 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e9807740 10 client.0 _ll_register_callbacks cb 0x5578415d5e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e49b5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e49b5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416ba110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41480/0 (socket says 172.21.15.92:41480) 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e49b5640 1 -- 172.21.15.92:0/3388015252 learned_addr learned my addr 172.21.15.92:0/3388015252 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8dffff640 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 0x5578416ba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e51b6640 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e49b5640 1 -- 172.21.15.92:0/3388015252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 msgr2=0x5578416ba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 0x5578416ba650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e49b5640 1 -- 172.21.15.92:0/3388015252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 msgr2=0x5578416c1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416c1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e49b5640 1 -- 172.21.15.92:0/3388015252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578415cfc60 con 0x5578416b0100 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.989+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416ba110 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8004270 tx=0x7fe8d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e84a9c3555b6ea server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8bffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8bffff640 1 -- 172.21.15.92:0/3388015252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8020930 con 0x5578416b0100 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8bffff640 1 -- 172.21.15.92:0/3388015252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe8d8020ad0 con 0x5578416b0100 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8e9807740 1 -- 172.21.15.92:0/3388015252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c8005500 con 0x5578416b0100 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8bffff640 1 -- 172.21.15.92:0/3388015252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8d8020cd0 con 0x5578416b0100 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8e9807740 10 client.16140 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8e9807740 1 -- 172.21.15.92:0/3388015252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557841329210 con 0x5578416b0100 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8bffff640 1 -- 172.21.15.92:0/3388015252 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c8005500 con 0x5578416b0100 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8bffff640 1 -- 172.21.15.92:0/3388015252 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe8d801f070 con 0x5578416b0100 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8e9807740 10 client.16140 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8e9807740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8e9807740 20 client.16140 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5578416c5fe0 0x5578416c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8e9807740 4 client.16140 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8e9807740 1 -- 172.21.15.92:0/3388015252 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557841622120 con 0x5578416c5fe0 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.990+0000 7fe8dffff640 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5578416c5fe0 0x5578416c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.991+0000 7fe8bffff640 10 client.16140 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.991+0000 7fe8dffff640 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5578416c5fe0 0x5578416c8410 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6918dfa41be08f7 server_cookie=66508850c90a49f6 in_seq=0 out_seq=0 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.991+0000 7fe8bffff640 1 -- 172.21.15.92:0/3388015252 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557841622120 con 0x5578416c5fe0 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.991+0000 7fe8bffff640 10 client.16140 handle_command_reply: tid=0 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 client.16140 shutdown 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 -- 172.21.15.92:0/3388015252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 msgr2=0x5578416ba110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416ba110 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8004270 tx=0x7fe8d8004ab0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 -- 172.21.15.92:0/3388015252 shutdown_connections 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x5578416c5fe0 0x5578416c8410 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 0x5578416ba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 --2- 172.21.15.92:0/3388015252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416ba110 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 -- 172.21.15.92:0/3388015252 >> 172.21.15.92:0/3388015252 conn(0x5578416ab670 msgr2=0x5578416ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8bffff640 0 client.16140 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 -- 172.21.15.92:0/3388015252 shutdown_connections 2024-03-19T02:02:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 1 -- 172.21.15.92:0/3388015252 wait complete. 2024-03-19T02:02:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:58.992+0000 7fe8e9807740 20 client.16140 trim_cache size 0 max 16384 2024-03-19T02:02:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.006+0000 7fe8e9807740 1 Processor -- start 2024-03-19T02:02:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.006+0000 7fe8e9807740 1 -- start start 2024-03-19T02:02:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.006+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 0x5578416ba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.006+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0af0 0x5578416aa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.006+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.006+0000 7fe8e9807740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01a8bd0 con 0x5578416b0af0 2024-03-19T02:02:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.006+0000 7fe8e9807740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01aa710 con 0x5578416b5c60 2024-03-19T02:02:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.006+0000 7fe8e9807740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01a92a0 con 0x5578416b0100 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0af0 0x5578416aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8e49b5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 0x5578416ba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0af0 0x5578416aa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41490/0 (socket says 172.21.15.92:41490) 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8dffff640 1 -- 172.21.15.92:0/1144809524 learned_addr learned my addr 172.21.15.92:0/1144809524 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8e51b6640 1 --2- 172.21.15.92:0/1144809524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8dffff640 1 -- 172.21.15.92:0/1144809524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 msgr2=0x5578416ba5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8dffff640 1 --2- 172.21.15.92:0/1144809524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 0x5578416ba5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8dffff640 1 -- 172.21.15.92:0/1144809524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 msgr2=0x5578416b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8dffff640 1 --2- 172.21.15.92:0/1144809524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8dffff640 1 -- 172.21.15.92:0/1144809524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578416a6ee0 con 0x5578416b0af0 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8dffff640 1 --2- 172.21.15.92:0/1144809524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0af0 0x5578416aa930 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009b20 tx=0x7fe8c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcbe30da6214552 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8d67fc640 1 -- 172.21.15.92:0/1144809524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c802f020 con 0x5578416b0af0 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8d67fc640 1 -- 172.21.15.92:0/1144809524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c8004040 con 0x5578416b0af0 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.007+0000 7fe8d67fc640 1 -- 172.21.15.92:0/1144809524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c802b540 con 0x5578416b0af0 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 -- 172.21.15.92:0/1144809524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0af0 msgr2=0x5578416aa930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1144809524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0af0 0x5578416aa930 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009b20 tx=0x7fe8c8011810 comp rx=0 tx=0).stop 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 -- 172.21.15.92:0/1144809524 shutdown_connections 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1144809524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1144809524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0af0 0x5578416aa930 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1144809524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0100 0x5578416ba5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 -- 172.21.15.92:0/1144809524 >> 172.21.15.92:0/1144809524 conn(0x5578416ab670 msgr2=0x5578416aa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 -- 172.21.15.92:0/1144809524 shutdown_connections 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 -- 172.21.15.92:0/1144809524 wait complete. 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 -- start start 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 Processor -- start 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 0x5578416bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01aa040 con 0x5578416b0100 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01a9970 con 0x5578416b5c60 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.008+0000 7fe8e9807740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01a6bb0 con 0x5578416b0af0 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e9807740 10 client.0 _ll_register_callbacks cb 0x7fe8d80092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:02:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e49b5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e49b5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416bec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41500/0 (socket says 172.21.15.92:41500) 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e49b5640 1 -- 172.21.15.92:0/1407650005 learned_addr learned my addr 172.21.15.92:0/1407650005 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8dffff640 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 0x5578416bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e51b6640 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e49b5640 1 -- 172.21.15.92:0/1407650005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 msgr2=0x5578416bf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 0x5578416bf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e49b5640 1 -- 172.21.15.92:0/1407650005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 msgr2=0x5578416c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e49b5640 1 -- 172.21.15.92:0/1407650005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578413f5620 con 0x5578416b0100 2024-03-19T02:02:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e49b5640 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416bec80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8010db0 tx=0x7fe8d8013a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a40b3b3b2dcf60 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8bffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8bffff640 1 -- 172.21.15.92:0/1407650005 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8003c80 con 0x5578416b0100 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e9807740 1 -- 172.21.15.92:0/1407650005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c8004040 con 0x5578416b0100 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8bffff640 1 -- 172.21.15.92:0/1407650005 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe8d8020a10 con 0x5578416b0100 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8bffff640 1 -- 172.21.15.92:0/1407650005 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8d8010430 con 0x5578416b0100 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.009+0000 7fe8e9807740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8e9807740 1 -- 172.21.15.92:0/1407650005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55784172c420 con 0x5578416b0100 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8bffff640 1 -- 172.21.15.92:0/1407650005 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c8004040 con 0x5578416b0100 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8bffff640 1 -- 172.21.15.92:0/1407650005 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe8d801f040 con 0x5578416b0100 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8e9807740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:02:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8e9807740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:02:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8e9807740 20 client.16146 populate_metadata read hostname 'smithi092' 2024-03-19T02:02:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55784172ce20 0x55784172f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:02:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8e9807740 4 client.16146 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:02:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8e9807740 1 -- 172.21.15.92:0/1407650005 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55784161c470 con 0x55784172ce20 2024-03-19T02:02:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8dffff640 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55784172ce20 0x55784172f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:02:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8bffff640 10 client.16146 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.010+0000 7fe8dffff640 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55784172ce20 0x55784172f240 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=207be1ecc86bf87b server_cookie=367f89e5046ec665 in_seq=0 out_seq=0 2024-03-19T02:02:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8bffff640 1 -- 172.21.15.92:0/1407650005 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55784161c470 con 0x55784172ce20 2024-03-19T02:02:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8bffff640 10 client.16146 handle_command_reply: tid=0 2024-03-19T02:02:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8e9807740 1 client.16146 shutdown 2024-03-19T02:02:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8e9807740 1 -- 172.21.15.92:0/1407650005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 msgr2=0x5578416bec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416bec80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8010db0 tx=0x7fe8d8013a90 comp rx=0 tx=0).stop 2024-03-19T02:02:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8e9807740 1 -- 172.21.15.92:0/1407650005 shutdown_connections 2024-03-19T02:02:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55784172ce20 0x55784172f240 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578416b5c60 0x5578416c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8bffff640 0 client.16146 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:02:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578416b0af0 0x5578416bf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8e9807740 1 --2- 172.21.15.92:0/1407650005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5578416b0100 0x5578416bec80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.017+0000 7fe8e9807740 1 -- 172.21.15.92:0/1407650005 >> 172.21.15.92:0/1407650005 conn(0x5578416ab670 msgr2=0x5578416ba3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e9807740 1 -- 172.21.15.92:0/1407650005 shutdown_connections 2024-03-19T02:02:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e9807740 1 -- 172.21.15.92:0/1407650005 wait complete. 2024-03-19T02:02:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e9807740 20 client.16146 trim_cache size 0 max 16384 2024-03-19T02:02:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:02:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:02:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:02:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:02:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 -- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe8cc0a28a0 msgr2=0x7fe8cc0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe8cc0a28a0 0x7fe8cc0a4d60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0101790 tx=0x7fe8d800ff30 comp rx=0 tx=0).stop 2024-03-19T02:02:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 -- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 msgr2=0x7fe8e019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:02:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 0x7fe8e019ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0005ab0 tx=0x7fe8d002c6c0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 -- 172.21.15.92:0/1289412649 shutdown_connections 2024-03-19T02:02:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e01022f0 0x7fe8e019ede0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fe8cc0a28a0 0x7fe8cc0a4d60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e0101930 0x7fe8e019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 --2- 172.21.15.92:0/1289412649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e0100740 0x7fe8e0069a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:02:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 -- 172.21.15.92:0/1289412649 >> 172.21.15.92:0/1289412649 conn(0x7fe8e00fbcb0 msgr2=0x7fe8e0104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:02:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 -- 172.21.15.92:0/1289412649 shutdown_connections 2024-03-19T02:02:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:02:59.018+0000 7fe8e6c19640 1 -- 172.21.15.92:0/1289412649 wait complete. 2024-03-19T02:02:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:02:58 smithi092 ceph-mon[62574]: pgmap v656: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:02:59.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:02:58 smithi144 ceph-mon[67199]: pgmap v656: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:02:59.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:02:58 smithi159 ceph-mon[67110]: pgmap v656: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-19T02:03:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:00 smithi092 ceph-mon[62574]: pgmap v657: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 18 op/s 2024-03-19T02:03:01.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:00 smithi144 ceph-mon[67199]: pgmap v657: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 18 op/s 2024-03-19T02:03:01.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:00 smithi159 ceph-mon[67110]: pgmap v657: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 18 op/s 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:499/600: open fd = 3 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:500/600: open fd = 3 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:501/600: open fd = 3 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.375 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:502/600: open fd = 3 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:503/600: open fd = 3 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:504/600: open fd = 3 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:505/600: open fd = 3 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.376 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:506/600: open fd = 3 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:507/600: open fd = 3 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:508/600: open fd = 3 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:509/600: open fd = 3 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.377 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:510/600: open fd = 3 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:511/600: open fd = 3 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:512/600: open fd = 3 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:513/600: open fd = 3 2024-03-19T02:03:02.378 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:514/600: open fd = 3 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:515/600: open fd = 3 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:516/600: open fd = 3 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.379 INFO:tasks.workunit.client.0.smithi092.stdout:517/600: open fd = 3 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:518/600: open fd = 3 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:519/600: open fd = 3 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:520/600: open fd = 3 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.380 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:521/600: open fd = 3 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:522/600: open fd = 3 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:523/600: open fd = 3 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:524/600: open fd = 3 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.381 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:525/600: open fd = 3 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:526/600: open fd = 3 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:527/600: open fd = 3 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:528/600: open fd = 3 2024-03-19T02:03:02.382 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:529/600: open fd = 3 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:530/600: open fd = 3 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:531/600: open fd = 3 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.383 INFO:tasks.workunit.client.0.smithi092.stdout:532/600: open fd = 3 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:533/600: open fd = 3 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:534/600: open fd = 3 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:535/600: open fd = 3 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.384 INFO:tasks.workunit.client.0.smithi092.stdout:536/600: open fd = 3 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:537/600: open fd = 3 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:538/600: open fd = 3 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:539/600: open fd = 3 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.385 INFO:tasks.workunit.client.0.smithi092.stdout:540/600: open fd = 3 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:541/600: open fd = 3 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:542/600: open fd = 3 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:543/600: open fd = 3 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.386 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:544/600: open fd = 3 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:545/600: open fd = 3 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:546/600: open fd = 3 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:547/600: open fd = 3 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.387 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:548/600: open fd = 3 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:549/600: open fd = 3 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:550/600: open fd = 3 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:551/600: open fd = 3 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.388 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:552/600: open fd = 3 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:553/600: open fd = 3 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:554/600: open fd = 3 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:555/600: open fd = 3 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.389 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:556/600: open fd = 3 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:557/600: open fd = 3 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:558/600: open fd = 3 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:559/600: open fd = 3 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.390 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.391 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:02.391 INFO:tasks.workunit.client.0.smithi092.stdout:560/600: open fd = 3 2024-03-19T02:03:02.391 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.391 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:02.391 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:02 smithi092 ceph-mon[62574]: pgmap v658: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-03-19T02:03:03.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:02 smithi144 ceph-mon[67199]: pgmap v658: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-03-19T02:03:03.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:02 smithi159 ceph-mon[67110]: pgmap v658: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-03-19T02:03:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:04 smithi092 ceph-mon[62574]: pgmap v659: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:05.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:04 smithi144 ceph-mon[67199]: pgmap v659: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:05.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:04 smithi159 ceph-mon[67110]: pgmap v659: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:06 smithi092 ceph-mon[62574]: pgmap v660: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:06 smithi144 ceph-mon[67199]: pgmap v660: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:07.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:06 smithi159 ceph-mon[67110]: pgmap v660: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:08 smithi092 ceph-mon[62574]: pgmap v661: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:03:09.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:08 smithi144 ceph-mon[67199]: pgmap v661: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:03:09.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:08 smithi159 ceph-mon[67110]: pgmap v661: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 16 op/s 2024-03-19T02:03:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:10 smithi092 ceph-mon[62574]: pgmap v662: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:10 smithi144 ceph-mon[67199]: pgmap v662: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:11.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:10 smithi159 ceph-mon[67110]: pgmap v662: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.105+0000 7fccf2c85640 1 -- 172.21.15.92:0/2589907450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 msgr2=0x7fccec107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.105+0000 7fccf2c85640 1 --2- 172.21.15.92:0/2589907450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 0x7fccec107060 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009900 tx=0x7fccd802bc30 comp rx=0 tx=0).stop 2024-03-19T02:03:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.105+0000 7fccf2c85640 1 -- 172.21.15.92:0/2589907450 shutdown_connections 2024-03-19T02:03:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.105+0000 7fccf2c85640 1 --2- 172.21.15.92:0/2589907450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccec1075a0 0x7fccec109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.105+0000 7fccf2c85640 1 --2- 172.21.15.92:0/2589907450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 0x7fccec107060 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.105+0000 7fccf2c85640 1 --2- 172.21.15.92:0/2589907450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccec102340 0x7fccec104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.105+0000 7fccf2c85640 1 -- 172.21.15.92:0/2589907450 >> 172.21.15.92:0/2589907450 conn(0x7fccec0fbcb0 msgr2=0x7fccec0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.105+0000 7fccf2c85640 1 -- 172.21.15.92:0/2589907450 shutdown_connections 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.105+0000 7fccf2c85640 1 -- 172.21.15.92:0/2589907450 wait complete. 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf2c85640 1 Processor -- start 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf2c85640 1 -- start start 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf2c85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccec102340 0x7fccec197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf2c85640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 0x7fccec198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf2c85640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccec1075a0 0x7fccec19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf2c85640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fccec10d4b0 con 0x7fccec104c70 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf2c85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccec10d330 con 0x7fccec102340 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf2c85640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fccec10d630 con 0x7fccec1075a0 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 0x7fccec198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf0a21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccec102340 0x7fccec197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 0x7fccec198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56776/0 (socket says 172.21.15.92:56776) 2024-03-19T02:03:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccebfff640 1 -- 172.21.15.92:0/4082169629 learned_addr learned my addr 172.21.15.92:0/4082169629 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.106+0000 7fccf1222640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccec1075a0 0x7fccec19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fccebfff640 1 -- 172.21.15.92:0/4082169629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccec1075a0 msgr2=0x7fccec19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fccebfff640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccec1075a0 0x7fccec19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fccebfff640 1 -- 172.21.15.92:0/4082169629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccec102340 msgr2=0x7fccec197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fccebfff640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccec102340 0x7fccec197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fccebfff640 1 -- 172.21.15.92:0/4082169629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec19f720 con 0x7fccec104c70 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fccebfff640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 0x7fccec198040 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000c00 tx=0x7fccd802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fcce9ffb640 1 -- 172.21.15.92:0/4082169629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8041070 con 0x7fccec104c70 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fcce9ffb640 1 -- 172.21.15.92:0/4082169629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd8004450 con 0x7fccec104c70 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fcce9ffb640 1 -- 172.21.15.92:0/4082169629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8034440 con 0x7fccec104c70 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fccf2c85640 1 -- 172.21.15.92:0/4082169629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec19f9b0 con 0x7fccec104c70 2024-03-19T02:03:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.107+0000 7fccf2c85640 1 -- 172.21.15.92:0/4082169629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec19fef0 con 0x7fccec104c70 2024-03-19T02:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.109+0000 7fcce9ffb640 1 -- 172.21.15.92:0/4082169629 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fccd8044070 con 0x7fccec104c70 2024-03-19T02:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf587a740 1 Processor -- start 2024-03-19T02:03:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf587a740 1 -- start start 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fcce9ffb640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fccdc0a2920 0x7fccdc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf587a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7510 0x55c8937c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf587a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 0x55c8937cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf587a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937cd070 0x55c8937cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf587a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c893727be0 con 0x55c8937c7510 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf587a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c8936d0960 con 0x55c8937c7f00 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf587a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c893723c80 con 0x55c8937cd070 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf0a21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7510 0x55c8937c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf1222640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fccdc0a2920 0x7fccdc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf0a21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7510 0x55c8937c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56780/0 (socket says 172.21.15.92:56780) 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf0a21640 1 -- 172.21.15.92:0/817840668 learned_addr learned my addr 172.21.15.92:0/817840668 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fcce9ffb640 1 -- 172.21.15.92:0/4082169629 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fccd80f0490 con 0x7fccec104c70 2024-03-19T02:03:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccebfff640 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 0x55c8937cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf1222640 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937cd070 0x55c8937cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf0a21640 1 -- 172.21.15.92:0/817840668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937cd070 msgr2=0x55c8937cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf0a21640 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937cd070 0x55c8937cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf0a21640 1 -- 172.21.15.92:0/817840668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 msgr2=0x55c8937cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf0a21640 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 0x55c8937cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccf0a21640 1 -- 172.21.15.92:0/817840668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8936c7d20 con 0x55c8937c7510 2024-03-19T02:03:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.110+0000 7fccebfff640 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 0x55c8937cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:03:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf0a21640 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7510 0x55c8937c7930 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009d20 tx=0x7fcce0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbb091959b95e07 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf1222640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fccdc0a2920 0x7fccdc0a4de0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcce4005fd0 tx=0x7fcce4009e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccd6ffd640 1 -- 172.21.15.92:0/817840668 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0019620 con 0x55c8937c7510 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccd6ffd640 1 -- 172.21.15.92:0/817840668 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce0019ca0 con 0x55c8937c7510 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccd6ffd640 1 -- 172.21.15.92:0/817840668 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0018cb0 con 0x55c8937c7510 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf587a740 1 -- 172.21.15.92:0/817840668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7510 msgr2=0x55c8937c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf587a740 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7510 0x55c8937c7930 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009d20 tx=0x7fcce0011810 comp rx=0 tx=0).stop 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf587a740 1 -- 172.21.15.92:0/817840668 shutdown_connections 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf587a740 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937cd070 0x55c8937cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf587a740 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 0x55c8937cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf587a740 1 --2- 172.21.15.92:0/817840668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7510 0x55c8937c7930 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf587a740 1 -- 172.21.15.92:0/817840668 >> 172.21.15.92:0/817840668 conn(0x55c8937be670 msgr2=0x55c8937bea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf587a740 1 -- 172.21.15.92:0/817840668 shutdown_connections 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.111+0000 7fccf587a740 1 -- 172.21.15.92:0/817840668 wait complete. 2024-03-19T02:03:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf587a740 1 -- start start 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf587a740 1 Processor -- start 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf587a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 0x55c8937d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf587a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 0x55c8937d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf587a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf587a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c8937d0020 con 0x55c8937cd070 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf587a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c8936e3db0 con 0x55c8937c7f00 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf587a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c8937d01a0 con 0x55c8937c7510 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf587a740 10 client.0 _ll_register_callbacks cb 0x55c8936e8e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf1222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf0a21640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 0x55c8937d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf1222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56782/0 (socket says 172.21.15.92:56782) 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf1222640 1 -- 172.21.15.92:0/389822486 learned_addr learned my addr 172.21.15.92:0/389822486 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccf0a21640 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 0x55c8937d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:55868/0 (socket says 172.21.15.92:55868) 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.112+0000 7fccebfff640 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 0x55c8937d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf1222640 1 -- 172.21.15.92:0/389822486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 msgr2=0x55c8937d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf1222640 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 0x55c8937d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf1222640 1 -- 172.21.15.92:0/389822486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 msgr2=0x55c8937d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf1222640 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 0x55c8937d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf1222640 1 -- 172.21.15.92:0/389822486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8936e2c60 con 0x55c8937cd070 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf1222640 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d8a40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcce400ffd0 tx=0x7fcce401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae8908d49339540 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fcccbfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fcccbfff640 1 -- 172.21.15.92:0/389822486 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4004b20 con 0x55c8937cd070 2024-03-19T02:03:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fcccbfff640 1 -- 172.21.15.92:0/389822486 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcce401c040 con 0x55c8937cd070 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf587a740 1 -- 172.21.15.92:0/389822486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce0019ca0 con 0x55c8937cd070 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fcccbfff640 1 -- 172.21.15.92:0/389822486 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcce4031050 con 0x55c8937cd070 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf587a740 10 client.16158 fetch_fsmap learned FSMap version 29 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf587a740 1 -- 172.21.15.92:0/389822486 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c89343c210 con 0x55c8937cd070 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fcccbfff640 1 -- 172.21.15.92:0/389822486 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce0019ca0 con 0x55c8937cd070 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fcccbfff640 1 -- 172.21.15.92:0/389822486 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcce4028070 con 0x55c8937cd070 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf587a740 10 client.16158 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf587a740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf587a740 20 client.16158 populate_metadata read hostname 'smithi092' 2024-03-19T02:03:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf587a740 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c8937dd390 0x55c8937df820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf587a740 4 client.16158 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.113+0000 7fccf587a740 1 -- 172.21.15.92:0/389822486 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c893735120 con 0x55c8937dd390 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.114+0000 7fccf0a21640 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c8937dd390 0x55c8937df820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.114+0000 7fcccbfff640 10 client.16158 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.114+0000 7fccf0a21640 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c8937dd390 0x55c8937df820 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1611ef348e975af7 server_cookie=9f5a474cc2977242 in_seq=0 out_seq=0 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fcccbfff640 1 -- 172.21.15.92:0/389822486 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c893735120 con 0x55c8937dd390 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fcccbfff640 10 client.16158 handle_command_reply: tid=0 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 client.16158 shutdown 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 -- 172.21.15.92:0/389822486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 msgr2=0x55c8937d8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d8a40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcce400ffd0 tx=0x7fcce401d000 comp rx=0 tx=0).stop 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 -- 172.21.15.92:0/389822486 shutdown_connections 2024-03-19T02:03:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c8937dd390 0x55c8937df820 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d8a40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fcccbfff640 0 client.16158 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7f00 0x55c8937d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 --2- 172.21.15.92:0/389822486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 0x55c8937d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 -- 172.21.15.92:0/389822486 >> 172.21.15.92:0/389822486 conn(0x55c8937be670 msgr2=0x55c8937bf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 -- 172.21.15.92:0/389822486 shutdown_connections 2024-03-19T02:03:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 1 -- 172.21.15.92:0/389822486 wait complete. 2024-03-19T02:03:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.115+0000 7fccf587a740 20 client.16158 trim_cache size 0 max 16384 2024-03-19T02:03:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf587a740 1 Processor -- start 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf587a740 1 -- start start 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf587a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 0x55c8937c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf587a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7f00 0x55c8937d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf587a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937cd070 0x55c8937e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf587a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1a69a0 con 0x55c8937c7f00 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf587a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1a84e0 con 0x55c8937cd070 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf587a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1a7070 con 0x55c8937c7510 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7f00 0x55c8937d8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccebfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7f00 0x55c8937d8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56796/0 (socket says 172.21.15.92:56796) 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccebfff640 1 -- 172.21.15.92:0/3338100807 learned_addr learned my addr 172.21.15.92:0/3338100807 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf1222640 1 --2- 172.21.15.92:0/3338100807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937cd070 0x55c8937e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccf0a21640 1 --2- 172.21.15.92:0/3338100807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 0x55c8937c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccebfff640 1 -- 172.21.15.92:0/3338100807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 msgr2=0x55c8937c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccebfff640 1 --2- 172.21.15.92:0/3338100807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 0x55c8937c1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccebfff640 1 -- 172.21.15.92:0/3338100807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937cd070 msgr2=0x55c8937e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccebfff640 1 --2- 172.21.15.92:0/3338100807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937cd070 0x55c8937e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.130+0000 7fccebfff640 1 -- 172.21.15.92:0/3338100807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c893508620 con 0x55c8937c7f00 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccebfff640 1 --2- 172.21.15.92:0/3338100807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7f00 0x55c8937d8a00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ee310 tx=0x7fccd8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6164e29ec75d44a server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccd67fc640 1 -- 172.21.15.92:0/3338100807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd804b040 con 0x55c8937c7f00 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccd67fc640 1 -- 172.21.15.92:0/3338100807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd8004c80 con 0x55c8937c7f00 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccd67fc640 1 -- 172.21.15.92:0/3338100807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80b95f0 con 0x55c8937c7f00 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccf587a740 1 -- 172.21.15.92:0/3338100807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7f00 msgr2=0x55c8937d8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccf587a740 1 --2- 172.21.15.92:0/3338100807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7f00 0x55c8937d8a00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ee310 tx=0x7fccd8005e50 comp rx=0 tx=0).stop 2024-03-19T02:03:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccf587a740 1 -- 172.21.15.92:0/3338100807 shutdown_connections 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccf587a740 1 --2- 172.21.15.92:0/3338100807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937cd070 0x55c8937e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccf587a740 1 --2- 172.21.15.92:0/3338100807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937c7f00 0x55c8937d8a00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccf587a740 1 --2- 172.21.15.92:0/3338100807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7510 0x55c8937c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccf587a740 1 -- 172.21.15.92:0/3338100807 >> 172.21.15.92:0/3338100807 conn(0x55c8937be670 msgr2=0x55c8937c18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccf587a740 1 -- 172.21.15.92:0/3338100807 shutdown_connections 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.131+0000 7fccf587a740 1 -- 172.21.15.92:0/3338100807 wait complete. 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf587a740 1 -- start start 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf587a740 1 Processor -- start 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf587a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7510 0x55c8937d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf587a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7f00 0x55c8937d68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf587a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf587a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1a7e10 con 0x55c8937cd070 2024-03-19T02:03:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf587a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1a7740 con 0x55c8937c7510 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf587a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1a4980 con 0x55c8937c7f00 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf587a740 10 client.0 _ll_register_callbacks cb 0x7fcce4009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf1222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf1222640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56812/0 (socket says 172.21.15.92:56812) 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf1222640 1 -- 172.21.15.92:0/2752053520 learned_addr learned my addr 172.21.15.92:0/2752053520 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccebfff640 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7f00 0x55c8937d68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf0a21640 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7510 0x55c8937d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf1222640 1 -- 172.21.15.92:0/2752053520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7f00 msgr2=0x55c8937d68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf1222640 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7f00 0x55c8937d68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf1222640 1 -- 172.21.15.92:0/2752053520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7510 msgr2=0x55c8937d6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf1222640 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7510 0x55c8937d6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.132+0000 7fccf1222640 1 -- 172.21.15.92:0/2752053520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8937d7eb0 con 0x55c8937cd070 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf1222640 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d78c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcce400fc20 tx=0x7fcce4004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85fa63d9dba402d server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fcccbfff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fcccbfff640 1 -- 172.21.15.92:0/2752053520 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4004ea0 con 0x55c8937cd070 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fcccbfff640 1 -- 172.21.15.92:0/2752053520 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcce401c040 con 0x55c8937cd070 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf587a740 1 -- 172.21.15.92:0/2752053520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd8004c80 con 0x55c8937cd070 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fcccbfff640 1 -- 172.21.15.92:0/2752053520 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcce4003970 con 0x55c8937cd070 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf587a740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf587a740 1 -- 172.21.15.92:0/2752053520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c893843a20 con 0x55c8937cd070 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fcccbfff640 1 -- 172.21.15.92:0/2752053520 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd8004c80 con 0x55c8937cd070 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fcccbfff640 1 -- 172.21.15.92:0/2752053520 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcce4028030 con 0x55c8937cd070 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf587a740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf587a740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf587a740 20 client.16164 populate_metadata read hostname 'smithi092' 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf587a740 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c8937dd390 0x55c893846430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf587a740 4 client.16164 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:03:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf587a740 1 -- 172.21.15.92:0/2752053520 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c89372f470 con 0x55c8937dd390 2024-03-19T02:03:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.133+0000 7fccf0a21640 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c8937dd390 0x55c893846430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.134+0000 7fcccbfff640 10 client.16164 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.134+0000 7fccf0a21640 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c8937dd390 0x55c893846430 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde950c349c83301 server_cookie=de6468f42abee8c9 in_seq=0 out_seq=0 2024-03-19T02:03:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.140+0000 7fcccbfff640 1 -- 172.21.15.92:0/2752053520 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c89372f470 con 0x55c8937dd390 2024-03-19T02:03:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.140+0000 7fcccbfff640 10 client.16164 handle_command_reply: tid=0 2024-03-19T02:03:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.140+0000 7fccf587a740 1 client.16164 shutdown 2024-03-19T02:03:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 -- 172.21.15.92:0/2752053520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 msgr2=0x55c8937d78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d78c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcce400fc20 tx=0x7fcce4004cb0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 -- 172.21.15.92:0/2752053520 shutdown_connections 2024-03-19T02:03:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55c8937dd390 0x55c893846430 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c8937cd070 0x55c8937d78c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8937c7f00 0x55c8937d68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fcccbfff640 0 client.16164 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 --2- 172.21.15.92:0/2752053520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8937c7510 0x55c8937d6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 -- 172.21.15.92:0/2752053520 >> 172.21.15.92:0/2752053520 conn(0x55c8937be670 msgr2=0x55c8937c2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 -- 172.21.15.92:0/2752053520 shutdown_connections 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 1 -- 172.21.15.92:0/2752053520 wait complete. 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf587a740 20 client.16164 trim_cache size 0 max 16384 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:03:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:03:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf2c85640 1 -- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fccdc0a2920 msgr2=0x7fccdc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf2c85640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fccdc0a2920 0x7fccdc0a4de0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcce4005fd0 tx=0x7fcce4009e00 comp rx=0 tx=0).stop 2024-03-19T02:03:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf2c85640 1 -- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 msgr2=0x7fccec198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.141+0000 7fccf2c85640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 0x7fccec198040 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000c00 tx=0x7fccd802c640 comp rx=0 tx=0).stop 2024-03-19T02:03:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.142+0000 7fccf2c85640 1 -- 172.21.15.92:0/4082169629 shutdown_connections 2024-03-19T02:03:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.142+0000 7fccf2c85640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fccdc0a2920 0x7fccdc0a4de0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.142+0000 7fccf2c85640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccec1075a0 0x7fccec19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.142+0000 7fccf2c85640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccec104c70 0x7fccec198040 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.142+0000 7fccf2c85640 1 --2- 172.21.15.92:0/4082169629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fccec102340 0x7fccec197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.142+0000 7fccf2c85640 1 -- 172.21.15.92:0/4082169629 >> 172.21.15.92:0/4082169629 conn(0x7fccec0fbcb0 msgr2=0x7fccec102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.142+0000 7fccf2c85640 1 -- 172.21.15.92:0/4082169629 shutdown_connections 2024-03-19T02:03:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:12.142+0000 7fccf2c85640 1 -- 172.21.15.92:0/4082169629 wait complete. 2024-03-19T02:03:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:12 smithi092 ceph-mon[62574]: pgmap v663: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:12 smithi144 ceph-mon[67199]: pgmap v663: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:13.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:12 smithi159 ceph-mon[67110]: pgmap v663: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:14 smithi092 ceph-mon[62574]: pgmap v664: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:03:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:14 smithi144 ceph-mon[67199]: pgmap v664: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:03:15.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:14 smithi159 ceph-mon[67110]: pgmap v664: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-03-19T02:03:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:16 smithi092 ceph-mon[62574]: pgmap v665: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T02:03:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:16 smithi144 ceph-mon[67199]: pgmap v665: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T02:03:17.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:16 smithi159 ceph-mon[67110]: pgmap v665: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-03-19T02:03:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:18 smithi092 ceph-mon[62574]: pgmap v666: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:03:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:18 smithi144 ceph-mon[67199]: pgmap v666: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:03:19.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:18 smithi159 ceph-mon[67110]: pgmap v666: 97 pgs: 97 active+clean; 3.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 16 op/s 2024-03-19T02:03:20.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:19 smithi092 ceph-mon[62574]: pgmap v667: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:20.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:19 smithi144 ceph-mon[67199]: pgmap v667: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:20.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:19 smithi159 ceph-mon[67110]: pgmap v667: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:20 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:03:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:20 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:03:21.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:20 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:03:22.346 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:22 smithi144 ceph-mon[67199]: pgmap v668: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:22.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:22 smithi159 ceph-mon[67110]: pgmap v668: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:22.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:22 smithi092 ceph-mon[62574]: pgmap v668: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:23.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:03:23.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:03:23.456 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:03:23.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:03:23.561 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.596 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:23.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:03:23.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:03:23.597 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:03:24.455 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:24 smithi159 ceph-mon[67110]: pgmap v669: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:24.461 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:24 smithi092 ceph-mon[62574]: pgmap v669: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:24.497 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:24 smithi144 ceph-mon[67199]: pgmap v669: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 -- 172.21.15.92:0/422175031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4101760 msgr2=0x7f95f4101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 --2- 172.21.15.92:0/422175031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4101760 0x7f95f4101be0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009900 tx=0x7f95dc02bc30 comp rx=0 tx=0).stop 2024-03-19T02:03:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 -- 172.21.15.92:0/422175031 shutdown_connections 2024-03-19T02:03:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 --2- 172.21.15.92:0/422175031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95f4102120 0x7f95f41089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 --2- 172.21.15.92:0/422175031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4101760 0x7f95f4101be0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 --2- 172.21.15.92:0/422175031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f4100570 0x7f95f4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 -- 172.21.15.92:0/422175031 >> 172.21.15.92:0/422175031 conn(0x7f95f40fbb00 msgr2=0x7f95f40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 -- 172.21.15.92:0/422175031 shutdown_connections 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 -- 172.21.15.92:0/422175031 wait complete. 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.229+0000 7f95f359e640 1 Processor -- start 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f359e640 1 -- start start 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f4100570 0x7f95f4069a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f359e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95f4101760 0x7f95f419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f359e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4102120 0x7f95f419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f359e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c200 con 0x7f95f4102120 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c080 con 0x7f95f4100570 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f359e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c380 con 0x7f95f4101760 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4102120 0x7f95f419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f4100570 0x7f95f4069a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95f4101760 0x7f95f419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f4100570 0x7f95f4069a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:53528/0 (socket says 172.21.15.92:53528) 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4102120 0x7f95f419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34792/0 (socket says 172.21.15.92:34792) 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95f4101760 0x7f95f419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:45686/0 (socket says 172.21.15.92:45686) 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f259c640 1 -- 172.21.15.92:0/2264816088 learned_addr learned my addr 172.21.15.92:0/2264816088 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f2d9d640 1 -- 172.21.15.92:0/2264816088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95f4101760 msgr2=0x7f95f419c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f2d9d640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95f4101760 0x7f95f419c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f2d9d640 1 -- 172.21.15.92:0/2264816088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f4100570 msgr2=0x7f95f4069a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f2d9d640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f4100570 0x7f95f4069a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f2d9d640 1 -- 172.21.15.92:0/2264816088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f419f5b0 con 0x7f95f4102120 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f259c640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f4100570 0x7f95f4069a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.230+0000 7f95f2d9d640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4102120 0x7f95f419ecf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009fd0 tx=0x7f95ec00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.231+0000 7f95ebfff640 1 -- 172.21.15.92:0/2264816088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00ce20 con 0x7f95f4102120 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.231+0000 7f95ebfff640 1 -- 172.21.15.92:0/2264816088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ec005020 con 0x7f95f4102120 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.231+0000 7f95f359e640 1 -- 172.21.15.92:0/2264816088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f419f8a0 con 0x7f95f4102120 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.231+0000 7f95ebfff640 1 -- 172.21.15.92:0/2264816088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec010720 con 0x7f95f4102120 2024-03-19T02:03:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.231+0000 7f95f359e640 1 -- 172.21.15.92:0/2264816088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f419fdc0 con 0x7f95f4102120 2024-03-19T02:03:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.233+0000 7f95ebfff640 1 -- 172.21.15.92:0/2264816088 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f95ec005540 con 0x7f95f4102120 2024-03-19T02:03:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95fb7ea740 1 Processor -- start 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95fb7ea740 1 -- start start 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252ef2e0 0x560a252ef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 0x560a252f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560a25253be0 con 0x560a252f4e40 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560a251fc960 con 0x560a252efcd0 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560a2524fc80 con 0x560a252ef2e0 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95f259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 0x560a252f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34794/0 (socket says 172.21.15.92:34794) 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95e9d9b640 1 -- 172.21.15.92:0/865297019 learned_addr learned my addr 172.21.15.92:0/865297019 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95f259c640 1 --2- 172.21.15.92:0/865297019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 0x560a252f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:53542/0 (socket says 172.21.15.92:53542) 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/865297019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252ef2e0 0x560a252ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95ebfff640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95e00a27f0 0x7f95e00a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95f259c640 1 -- 172.21.15.92:0/865297019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252ef2e0 msgr2=0x560a252ef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95f259c640 1 --2- 172.21.15.92:0/865297019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252ef2e0 0x560a252ef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95f259c640 1 -- 172.21.15.92:0/865297019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 msgr2=0x560a252f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.234+0000 7f95f259c640 1 --2- 172.21.15.92:0/865297019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.235+0000 7f95f2d9d640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95e00a27f0 0x7f95e00a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.235+0000 7f95f259c640 1 -- 172.21.15.92:0/865297019 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a251f3d20 con 0x560a252efcd0 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.235+0000 7f95ebfff640 1 -- 172.21.15.92:0/2264816088 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f95ec0cefc0 con 0x7f95f4102120 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.235+0000 7f95f259c640 1 --2- 172.21.15.92:0/865297019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 0x560a252f47d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95e40114c0 tx=0x7f95e4011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb028286379ac85 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.235+0000 7f95f2d9d640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95e00a27f0 0x7f95e00a4cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f95ec09b9a0 tx=0x7f95ec0667d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.235+0000 7f95e8d99640 1 -- 172.21.15.92:0/865297019 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4005020 con 0x560a252efcd0 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.235+0000 7f95e8d99640 1 -- 172.21.15.92:0/865297019 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e40047a0 con 0x560a252efcd0 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.235+0000 7f95e8d99640 1 -- 172.21.15.92:0/865297019 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e400f8c0 con 0x560a252efcd0 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 -- 172.21.15.92:0/865297019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 msgr2=0x560a252f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/865297019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 0x560a252f47d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95e40114c0 tx=0x7f95e4011980 comp rx=0 tx=0).stop 2024-03-19T02:03:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 -- 172.21.15.92:0/865297019 shutdown_connections 2024-03-19T02:03:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/865297019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/865297019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 0x560a252f47d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/865297019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252ef2e0 0x560a252ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 -- 172.21.15.92:0/865297019 >> 172.21.15.92:0/865297019 conn(0x560a252ea850 msgr2=0x560a252eac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 -- 172.21.15.92:0/865297019 shutdown_connections 2024-03-19T02:03:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 -- 172.21.15.92:0/865297019 wait complete. 2024-03-19T02:03:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.236+0000 7f95fb7ea740 1 -- start start 2024-03-19T02:03:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95fb7ea740 1 Processor -- start 2024-03-19T02:03:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 0x560a252f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 0x560a253008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560a252f7df0 con 0x560a252ef2e0 2024-03-19T02:03:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560a2520fdb0 con 0x560a252f4e40 2024-03-19T02:03:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560a252f7f70 con 0x560a252efcd0 2024-03-19T02:03:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95fb7ea740 10 client.0 _ll_register_callbacks cb 0x560a25214e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:03:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34796/0 (socket says 172.21.15.92:34796) 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 -- 172.21.15.92:0/649108839 learned_addr learned my addr 172.21.15.92:0/649108839 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95f259c640 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 0x560a252f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 0x560a253008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 -- 172.21.15.92:0/649108839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 msgr2=0x560a252f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 0x560a252f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 -- 172.21.15.92:0/649108839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 msgr2=0x560a253008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 0x560a253008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 -- 172.21.15.92:0/649108839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2520ec60 con 0x560a252ef2e0 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f9320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009230 tx=0x7f95dc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc45ba6bda9099a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95d9ffb640 1 -- 172.21.15.92:0/649108839 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc003a00 con 0x560a252ef2e0 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.237+0000 7f95d9ffb640 1 -- 172.21.15.92:0/649108839 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f95dc003ba0 con 0x560a252ef2e0 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95fb7ea740 1 -- 172.21.15.92:0/649108839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e40047a0 con 0x560a252ef2e0 2024-03-19T02:03:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95d9ffb640 1 -- 172.21.15.92:0/649108839 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95dc055050 con 0x560a252ef2e0 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95fb7ea740 10 client.16176 fetch_fsmap learned FSMap version 29 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95fb7ea740 1 -- 172.21.15.92:0/649108839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a24f68210 con 0x560a252ef2e0 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95d9ffb640 1 -- 172.21.15.92:0/649108839 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e40047a0 con 0x560a252ef2e0 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95d9ffb640 1 -- 172.21.15.92:0/649108839 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f95dc04c070 con 0x560a252ef2e0 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95fb7ea740 10 client.16176 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95fb7ea740 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95fb7ea740 20 client.16176 populate_metadata read hostname 'smithi092' 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560a25305130 0x560a25307560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95fb7ea740 4 client.16176 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95fb7ea740 1 -- 172.21.15.92:0/649108839 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a25261120 con 0x560a25305130 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.238+0000 7f95f259c640 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560a25305130 0x560a25307560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.239+0000 7f95d9ffb640 10 client.16176 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.239+0000 7f95f259c640 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560a25305130 0x560a25307560 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240d51bd3cadffe9 server_cookie=7c098c143d72c2e7 in_seq=0 out_seq=0 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95d9ffb640 1 -- 172.21.15.92:0/649108839 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560a25261120 con 0x560a25305130 2024-03-19T02:03:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95d9ffb640 10 client.16176 handle_command_reply: tid=0 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 client.16176 shutdown 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 -- 172.21.15.92:0/649108839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 msgr2=0x560a252f9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f9320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009230 tx=0x7f95dc02bc30 comp rx=0 tx=0).stop 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 -- 172.21.15.92:0/649108839 shutdown_connections 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560a25305130 0x560a25307560 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 0x560a253008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95d9ffb640 0 client.16176 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 0x560a252f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/649108839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f9320 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 -- 172.21.15.92:0/649108839 >> 172.21.15.92:0/649108839 conn(0x560a252ea850 msgr2=0x560a252e9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 -- 172.21.15.92:0/649108839 shutdown_connections 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 1 -- 172.21.15.92:0/649108839 wait complete. 2024-03-19T02:03:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.240+0000 7f95fb7ea740 20 client.16176 trim_cache size 0 max 16384 2024-03-19T02:03:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95fb7ea740 1 Processor -- start 2024-03-19T02:03:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95fb7ea740 1 -- start start 2024-03-19T02:03:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 0x560a25300860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 0x560a252ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41a8b60 con 0x560a252ef2e0 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41aa6a0 con 0x560a252f4e40 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41a9230 con 0x560a252efcd0 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34800/0 (socket says 172.21.15.92:34800) 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 -- 172.21.15.92:0/1929743880 learned_addr learned my addr 172.21.15.92:0/1929743880 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95f259c640 1 --2- 172.21.15.92:0/1929743880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 0x560a25300860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/1929743880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 0x560a252ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 -- 172.21.15.92:0/1929743880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 msgr2=0x560a25300860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/1929743880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 0x560a25300860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 -- 172.21.15.92:0/1929743880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 msgr2=0x560a252ee1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/1929743880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 0x560a252ee1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 -- 172.21.15.92:0/1929743880 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a25034620 con 0x560a252ef2e0 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/1929743880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f97c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95dc001eb0 tx=0x7f95dc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21eeecfb127e9f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.262+0000 7f95dbfff640 1 -- 172.21.15.92:0/1929743880 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc041bd0 con 0x560a252ef2e0 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95dbfff640 1 -- 172.21.15.92:0/1929743880 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95dc041d70 con 0x560a252ef2e0 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95dbfff640 1 -- 172.21.15.92:0/1929743880 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc04e6d0 con 0x560a252ef2e0 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1929743880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 msgr2=0x560a252f97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1929743880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f97c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95dc001eb0 tx=0x7f95dc005fd0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1929743880 shutdown_connections 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1929743880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252f4e40 0x560a252ee1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1929743880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252efcd0 0x560a25300860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1929743880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252ef2e0 0x560a252f97c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1929743880 >> 172.21.15.92:0/1929743880 conn(0x560a252ea850 msgr2=0x560a252edab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1929743880 shutdown_connections 2024-03-19T02:03:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.263+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1929743880 wait complete. 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95fb7ea740 1 -- start start 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95fb7ea740 1 Processor -- start 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252ef2e0 0x560a252fdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 0x560a252fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95fb7ea740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252ff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41a9fd0 con 0x560a252f4e40 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41a9900 con 0x560a252efcd0 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95fb7ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41a6b40 con 0x560a252ef2e0 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95fb7ea740 10 client.0 _ll_register_callbacks cb 0x7f95dc0041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95e9d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252ff170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34812/0 (socket says 172.21.15.92:34812) 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95e9d9b640 1 -- 172.21.15.92:0/1958625955 learned_addr learned my addr 172.21.15.92:0/1958625955 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95e9d9b640 1 -- 172.21.15.92:0/1958625955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252ef2e0 msgr2=0x560a252fdc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T02:03:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252ef2e0 0x560a252fdc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95e9d9b640 1 -- 172.21.15.92:0/1958625955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 msgr2=0x560a252fe1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 0x560a252fe1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95e9d9b640 1 -- 172.21.15.92:0/1958625955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a252ff760 con 0x560a252f4e40 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95e9d9b640 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252ff170 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f95dc036040 tx=0x7f95dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddff2300baad157e server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.264+0000 7f95d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95d9ffb640 1 -- 172.21.15.92:0/1958625955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc05f070 con 0x560a252f4e40 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95d9ffb640 1 -- 172.21.15.92:0/1958625955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f95dc04e470 con 0x560a252f4e40 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1958625955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95dc041d70 con 0x560a252f4e40 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95d9ffb640 1 -- 172.21.15.92:0/1958625955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95dc04e670 con 0x560a252f4e40 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95fb7ea740 10 client.16182 fetch_fsmap learned FSMap version 29 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1958625955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a25300a70 con 0x560a252f4e40 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95d9ffb640 1 -- 172.21.15.92:0/1958625955 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95dc041d70 con 0x560a252f4e40 2024-03-19T02:03:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95d9ffb640 1 -- 172.21.15.92:0/1958625955 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f95dc062030 con 0x560a252f4e40 2024-03-19T02:03:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95fb7ea740 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:03:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95fb7ea740 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:03:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95fb7ea740 20 client.16182 populate_metadata read hostname 'smithi092' 2024-03-19T02:03:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560a2536baf0 0x560a2536df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95fb7ea740 4 client.16182 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:03:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1958625955 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a2525b470 con 0x560a2536baf0 2024-03-19T02:03:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.265+0000 7f95f259c640 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560a2536baf0 0x560a2536df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.266+0000 7f95f259c640 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560a2536baf0 0x560a2536df10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c5c808ec887178d server_cookie=306159b4cd0c423f in_seq=0 out_seq=0 2024-03-19T02:03:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.266+0000 7f95d9ffb640 10 client.16182 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95d9ffb640 1 -- 172.21.15.92:0/1958625955 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a2525b470 con 0x560a2536baf0 2024-03-19T02:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95d9ffb640 10 client.16182 handle_command_reply: tid=0 2024-03-19T02:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 client.16182 shutdown 2024-03-19T02:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1958625955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 msgr2=0x560a252ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252ff170 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f95dc036040 tx=0x7f95dc000bc0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1958625955 shutdown_connections 2024-03-19T02:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x560a2536baf0 0x560a2536df10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560a252f4e40 0x560a252ff170 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95d9ffb640 0 client.16182 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560a252efcd0 0x560a252fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 --2- 172.21.15.92:0/1958625955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a252ef2e0 0x560a252fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1958625955 >> 172.21.15.92:0/1958625955 conn(0x560a252ea850 msgr2=0x560a252ee340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1958625955 shutdown_connections 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.273+0000 7f95fb7ea740 1 -- 172.21.15.92:0/1958625955 wait complete. 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95fb7ea740 20 client.16182 trim_cache size 0 max 16384 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:03:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:03:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 -- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95e00a27f0 msgr2=0x7f95e00a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95e00a27f0 0x7f95e00a4cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f95ec09b9a0 tx=0x7f95ec0667d0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 -- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4102120 msgr2=0x7f95f419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4102120 0x7f95f419ecf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009fd0 tx=0x7f95ec00ef10 comp rx=0 tx=0).stop 2024-03-19T02:03:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 -- 172.21.15.92:0/2264816088 shutdown_connections 2024-03-19T02:03:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95f4102120 0x7f95f419ecf0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f95e00a27f0 0x7f95e00a4cb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95f4101760 0x7f95f419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 --2- 172.21.15.92:0/2264816088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f4100570 0x7f95f4069a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 -- 172.21.15.92:0/2264816088 >> 172.21.15.92:0/2264816088 conn(0x7f95f40fbb00 msgr2=0x7f95f4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 -- 172.21.15.92:0/2264816088 shutdown_connections 2024-03-19T02:03:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:25.274+0000 7f95f359e640 1 -- 172.21.15.92:0/2264816088 wait complete. 2024-03-19T02:03:26.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:26 smithi159 ceph-mon[67110]: pgmap v670: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.3 KiB/s wr, 15 op/s 2024-03-19T02:03:27.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:26 smithi092 ceph-mon[62574]: pgmap v670: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.3 KiB/s wr, 15 op/s 2024-03-19T02:03:27.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:26 smithi144 ceph-mon[67199]: pgmap v670: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.3 KiB/s wr, 15 op/s 2024-03-19T02:03:28.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:28 smithi159 ceph-mon[67110]: pgmap v671: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-03-19T02:03:29.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:28 smithi092 ceph-mon[62574]: pgmap v671: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-03-19T02:03:29.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:28 smithi144 ceph-mon[67199]: pgmap v671: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-03-19T02:03:30.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:30 smithi159 ceph-mon[67110]: pgmap v672: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:03:31.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:30 smithi092 ceph-mon[62574]: pgmap v672: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:03:31.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:30 smithi144 ceph-mon[67199]: pgmap v672: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 20 op/s 2024-03-19T02:03:32.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:32 smithi159 ceph-mon[67110]: pgmap v673: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T02:03:33.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:32 smithi092 ceph-mon[62574]: pgmap v673: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T02:03:33.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:32 smithi144 ceph-mon[67199]: pgmap v673: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-19T02:03:34.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:34 smithi159 ceph-mon[67110]: pgmap v674: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:35.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:34 smithi092 ceph-mon[62574]: pgmap v674: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:35.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:34 smithi144 ceph-mon[67199]: pgmap v674: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-03-19T02:03:36.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:36 smithi159 ceph-mon[67110]: pgmap v675: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:37.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:36 smithi092 ceph-mon[62574]: pgmap v675: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:37.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:36 smithi144 ceph-mon[67199]: pgmap v675: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-03-19T02:03:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.363+0000 7f193e122640 1 -- 172.21.15.92:0/1420590170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1938101930 msgr2=0x7f1938101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.363+0000 7f193e122640 1 --2- 172.21.15.92:0/1420590170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1938101930 0x7f1938101db0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f192c009900 tx=0x7f192c02bc30 comp rx=0 tx=0).stop 2024-03-19T02:03:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.363+0000 7f193e122640 1 -- 172.21.15.92:0/1420590170 shutdown_connections 2024-03-19T02:03:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.363+0000 7f193e122640 1 --2- 172.21.15.92:0/1420590170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19381022f0 0x7f1938108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.363+0000 7f193e122640 1 --2- 172.21.15.92:0/1420590170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1938101930 0x7f1938101db0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.363+0000 7f193e122640 1 --2- 172.21.15.92:0/1420590170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1938100740 0x7f1938100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.363+0000 7f193e122640 1 -- 172.21.15.92:0/1420590170 >> 172.21.15.92:0/1420590170 conn(0x7f19380fbcb0 msgr2=0x7f19380fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.363+0000 7f193e122640 1 -- 172.21.15.92:0/1420590170 shutdown_connections 2024-03-19T02:03:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.363+0000 7f193e122640 1 -- 172.21.15.92:0/1420590170 wait complete. 2024-03-19T02:03:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f193e122640 1 Processor -- start 2024-03-19T02:03:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f193e122640 1 -- start start 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f193e122640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1938100740 0x7f1938197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f193e122640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1938101930 0x7f1938197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f193e122640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19381022f0 0x7f193819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f193e122640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f193810c3a0 con 0x7f19381022f0 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f193e122640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f193810c220 con 0x7f1938101930 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f193e122640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f193810c520 con 0x7f1938100740 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f1937fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19381022f0 0x7f193819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f1937fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19381022f0 0x7f193819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42206/0 (socket says 172.21.15.92:42206) 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f19377fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1938100740 0x7f1938197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f1937fff640 1 -- 172.21.15.92:0/3091967893 learned_addr learned my addr 172.21.15.92:0/3091967893 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.364+0000 7f1936ffd640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1938101930 0x7f1938197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1937fff640 1 -- 172.21.15.92:0/3091967893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1938100740 msgr2=0x7f1938197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1937fff640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1938100740 0x7f1938197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1937fff640 1 -- 172.21.15.92:0/3091967893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1938101930 msgr2=0x7f1938197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1937fff640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1938101930 0x7f1938197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1937fff640 1 -- 172.21.15.92:0/3091967893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193819f6d0 con 0x7f19381022f0 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1936ffd640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1938101930 0x7f1938197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1937fff640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19381022f0 0x7f193819f060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f193000eb10 tx=0x7f193000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1934ff9640 1 -- 172.21.15.92:0/3091967893 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930004b20 con 0x7f19381022f0 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1934ff9640 1 -- 172.21.15.92:0/3091967893 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1930004cc0 con 0x7f19381022f0 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f193e122640 1 -- 172.21.15.92:0/3091967893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193819f9c0 con 0x7f19381022f0 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f1934ff9640 1 -- 172.21.15.92:0/3091967893 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930010760 con 0x7f19381022f0 2024-03-19T02:03:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.365+0000 7f193e122640 1 -- 172.21.15.92:0/3091967893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193819fe90 con 0x7f19381022f0 2024-03-19T02:03:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.368+0000 7f1934ff9640 1 -- 172.21.15.92:0/3091967893 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f1930005020 con 0x7f19381022f0 2024-03-19T02:03:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.368+0000 7f1940dec740 1 Processor -- start 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.368+0000 7f1940dec740 1 -- start start 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.368+0000 7f1934ff9640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f19200a28a0 0x7f19200a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.368+0000 7f1940dec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bbf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.368+0000 7f1940dec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bc45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.368+0000 7f1940dec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bc4c60 0x55dee9bc70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.368+0000 7f1940dec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dee9b23be0 con 0x55dee9bc4c60 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f1940dec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dee9acc960 con 0x55dee9bbfaf0 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f1940dec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dee9b1fc80 con 0x55dee9bbf100 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f1936ffd640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f19200a28a0 0x7f19200a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f1934ff9640 1 -- 172.21.15.92:0/3091967893 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f19300cefc0 con 0x7f19381022f0 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f19377fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bbf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f19377fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bbf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.92:53894/0 (socket says 172.21.15.92:53894) 2024-03-19T02:03:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f19377fe640 1 -- 172.21.15.92:0/1786643916 learned_addr learned my addr 172.21.15.92:0/1786643916 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f1936ffd640 1 --2- 172.21.15.92:0/1786643916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bc45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f1937fff640 1 --2- 172.21.15.92:0/1786643916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bc4c60 0x55dee9bc70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f19377fe640 1 -- 172.21.15.92:0/1786643916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 msgr2=0x55dee9bc45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f19377fe640 1 --2- 172.21.15.92:0/1786643916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bc45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f19377fe640 1 -- 172.21.15.92:0/1786643916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bc4c60 msgr2=0x55dee9bc70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f19377fe640 1 --2- 172.21.15.92:0/1786643916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bc4c60 0x55dee9bc70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f19377fe640 1 -- 172.21.15.92:0/1786643916 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee9ac3d20 con 0x55dee9bbf100 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f1936ffd640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f19200a28a0 0x7f19200a4d60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f192c02c7b0 tx=0x7f192c02cdc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f19377fe640 1 --2- 172.21.15.92:0/1786643916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bbf520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1928009ca0 tx=0x7f1928011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e770d6bf24858fbf server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f191dffb640 1 -- 172.21.15.92:0/1786643916 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928004220 con 0x55dee9bbf100 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f191dffb640 1 -- 172.21.15.92:0/1786643916 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19280043c0 con 0x55dee9bbf100 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.369+0000 7f191dffb640 1 -- 172.21.15.92:0/1786643916 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19280188e0 con 0x55dee9bbf100 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 -- 172.21.15.92:0/1786643916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 msgr2=0x55dee9bbf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 --2- 172.21.15.92:0/1786643916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bbf520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1928009ca0 tx=0x7f1928011810 comp rx=0 tx=0).stop 2024-03-19T02:03:38.377 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 -- 172.21.15.92:0/1786643916 shutdown_connections 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 --2- 172.21.15.92:0/1786643916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bc4c60 0x55dee9bc70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 --2- 172.21.15.92:0/1786643916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bc45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 --2- 172.21.15.92:0/1786643916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bbf520 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 -- 172.21.15.92:0/1786643916 >> 172.21.15.92:0/1786643916 conn(0x55dee9bba670 msgr2=0x55dee9bbaa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 -- 172.21.15.92:0/1786643916 shutdown_connections 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 -- 172.21.15.92:0/1786643916 wait complete. 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.370+0000 7f1940dec740 1 -- start start 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1940dec740 1 Processor -- start 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1940dec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1940dec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bc9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1940dec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 0x55dee9bd05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1940dec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dee9bc7c10 con 0x55dee9bbf100 2024-03-19T02:03:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1940dec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dee9adfdb0 con 0x55dee9bbfaf0 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1940dec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dee9bc7d90 con 0x55dee9bc4c60 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1940dec740 10 client.0 _ll_register_callbacks cb 0x55dee9ae4e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f19377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f19377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42224/0 (socket says 172.21.15.92:42224) 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f19377fe640 1 -- 172.21.15.92:0/2713010480 learned_addr learned my addr 172.21.15.92:0/2713010480 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1937fff640 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 0x55dee9bd05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1936ffd640 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bc9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f19377fe640 1 -- 172.21.15.92:0/2713010480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 msgr2=0x55dee9bd05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f19377fe640 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 0x55dee9bd05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f19377fe640 1 -- 172.21.15.92:0/2713010480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 msgr2=0x55dee9bc9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f19377fe640 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bc9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f19377fe640 1 -- 172.21.15.92:0/2713010480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee9adec60 con 0x55dee9bbf100 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f19377fe640 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc9040 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2601231f914defe3 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.371+0000 7f1916ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:03:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1916ffd640 1 -- 172.21.15.92:0/2713010480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928027020 con 0x55dee9bbf100 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1916ffd640 1 -- 172.21.15.92:0/2713010480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f19280035f0 con 0x55dee9bbf100 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1940dec740 1 -- 172.21.15.92:0/2713010480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19280043c0 con 0x55dee9bbf100 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1916ffd640 1 -- 172.21.15.92:0/2713010480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f192801e040 con 0x55dee9bbf100 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1940dec740 10 client.16194 fetch_fsmap learned FSMap version 29 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1940dec740 1 -- 172.21.15.92:0/2713010480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dee9838210 con 0x55dee9bbf100 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1916ffd640 1 -- 172.21.15.92:0/2713010480 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19280043c0 con 0x55dee9bbf100 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1916ffd640 1 -- 172.21.15.92:0/2713010480 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1928004b00 con 0x55dee9bbf100 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1940dec740 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1940dec740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1940dec740 20 client.16194 populate_metadata read hostname 'smithi092' 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1940dec740 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55dee9bd4c40 0x55dee9bd7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1940dec740 4 client.16194 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1940dec740 1 -- 172.21.15.92:0/2713010480 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dee9b31120 con 0x55dee9bd4c40 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.372+0000 7f1936ffd640 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55dee9bd4c40 0x55dee9bd7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.373+0000 7f1916ffd640 10 client.16194 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.373+0000 7f1936ffd640 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55dee9bd4c40 0x55dee9bd7030 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f693f3b6fca28c2 server_cookie=62d2ab565e93f4e in_seq=0 out_seq=0 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1916ffd640 1 -- 172.21.15.92:0/2713010480 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dee9b31120 con 0x55dee9bd4c40 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1916ffd640 10 client.16194 handle_command_reply: tid=0 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 client.16194 shutdown 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 -- 172.21.15.92:0/2713010480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 msgr2=0x55dee9bc9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc9040 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928005250 comp rx=0 tx=0).stop 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 -- 172.21.15.92:0/2713010480 shutdown_connections 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55dee9bd4c40 0x55dee9bd7030 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 0x55dee9bd05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bc9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1916ffd640 0 client.16194 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 --2- 172.21.15.92:0/2713010480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc9040 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 -- 172.21.15.92:0/2713010480 >> 172.21.15.92:0/2713010480 conn(0x55dee9bba670 msgr2=0x55dee9bbb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 -- 172.21.15.92:0/2713010480 shutdown_connections 2024-03-19T02:03:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 1 -- 172.21.15.92:0/2713010480 wait complete. 2024-03-19T02:03:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.374+0000 7f1940dec740 20 client.16194 trim_cache size 0 max 16384 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1940dec740 1 Processor -- start 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1940dec740 1 -- start start 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1940dec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1940dec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bd04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1940dec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 0x55dee9bc8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1940dec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19381a8c40 con 0x55dee9bbf100 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1940dec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19381aa780 con 0x55dee9bbfaf0 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1940dec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19381a9310 con 0x55dee9bc4c60 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f19377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f19377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42232/0 (socket says 172.21.15.92:42232) 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f19377fe640 1 -- 172.21.15.92:0/3847513063 learned_addr learned my addr 172.21.15.92:0/3847513063 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1936ffd640 1 --2- 172.21.15.92:0/3847513063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bd04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f1937fff640 1 --2- 172.21.15.92:0/3847513063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 0x55dee9bc8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f19377fe640 1 -- 172.21.15.92:0/3847513063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 msgr2=0x55dee9bc8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f19377fe640 1 --2- 172.21.15.92:0/3847513063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 0x55dee9bc8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f19377fe640 1 -- 172.21.15.92:0/3847513063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 msgr2=0x55dee9bd04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f19377fe640 1 --2- 172.21.15.92:0/3847513063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bd04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f19377fe640 1 -- 172.21.15.92:0/3847513063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee9904620 con 0x55dee9bbf100 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.389+0000 7f19377fe640 1 --2- 172.21.15.92:0/3847513063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc8660 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1928011850 tx=0x7f1928001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e26715863220b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f191d7fa640 1 -- 172.21.15.92:0/3847513063 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928027070 con 0x55dee9bbf100 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f191d7fa640 1 -- 172.21.15.92:0/3847513063 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1928005d30 con 0x55dee9bbf100 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f191d7fa640 1 -- 172.21.15.92:0/3847513063 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928018e70 con 0x55dee9bbf100 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f1940dec740 1 -- 172.21.15.92:0/3847513063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 msgr2=0x55dee9bc8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f1940dec740 1 --2- 172.21.15.92:0/3847513063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc8660 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1928011850 tx=0x7f1928001e50 comp rx=0 tx=0).stop 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f1940dec740 1 -- 172.21.15.92:0/3847513063 shutdown_connections 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f1940dec740 1 --2- 172.21.15.92:0/3847513063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bc4c60 0x55dee9bc8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f1940dec740 1 --2- 172.21.15.92:0/3847513063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bbfaf0 0x55dee9bd04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f1940dec740 1 --2- 172.21.15.92:0/3847513063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbf100 0x55dee9bc8660 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f1940dec740 1 -- 172.21.15.92:0/3847513063 >> 172.21.15.92:0/3847513063 conn(0x55dee9bba670 msgr2=0x55dee9bbe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f1940dec740 1 -- 172.21.15.92:0/3847513063 shutdown_connections 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.390+0000 7f1940dec740 1 -- 172.21.15.92:0/3847513063 wait complete. 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1940dec740 1 -- start start 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1940dec740 1 Processor -- start 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1940dec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bcdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1940dec740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbfaf0 0x55dee9bce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1940dec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bc4c60 0x55dee9bcf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1940dec740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19381aa0b0 con 0x55dee9bbfaf0 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1940dec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19381a99e0 con 0x55dee9bc4c60 2024-03-19T02:03:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1940dec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19381a6c20 con 0x55dee9bbf100 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1940dec740 10 client.0 _ll_register_callbacks cb 0x7f19300057d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1936ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbfaf0 0x55dee9bce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1936ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbfaf0 0x55dee9bce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42238/0 (socket says 172.21.15.92:42238) 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1936ffd640 1 -- 172.21.15.92:0/471599209 learned_addr learned my addr 172.21.15.92:0/471599209 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f19377fe640 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bcdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.391+0000 7f1937fff640 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bc4c60 0x55dee9bcf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1936ffd640 1 -- 172.21.15.92:0/471599209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 msgr2=0x55dee9bcdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1936ffd640 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bcdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1936ffd640 1 -- 172.21.15.92:0/471599209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bc4c60 msgr2=0x55dee9bcf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1936ffd640 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bc4c60 0x55dee9bcf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1936ffd640 1 -- 172.21.15.92:0/471599209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee9bcf870 con 0x55dee9bbfaf0 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1936ffd640 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbfaf0 0x55dee9bce270 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f192c036040 tx=0x7f192c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3eb1a54b077753 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1916ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-19T02:03:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1916ffd640 1 -- 172.21.15.92:0/471599209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c059070 con 0x55dee9bbfaf0 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1916ffd640 1 -- 172.21.15.92:0/471599209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f192c005470 con 0x55dee9bbfaf0 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1940dec740 1 -- 172.21.15.92:0/471599209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1928005d30 con 0x55dee9bbfaf0 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1916ffd640 1 -- 172.21.15.92:0/471599209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f192c065050 con 0x55dee9bbfaf0 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1940dec740 10 client.16200 fetch_fsmap learned FSMap version 29 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1940dec740 1 -- 172.21.15.92:0/471599209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dee9c3b150 con 0x55dee9bbfaf0 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1916ffd640 1 -- 172.21.15.92:0/471599209 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1928005d30 con 0x55dee9bbfaf0 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1916ffd640 1 -- 172.21.15.92:0/471599209 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f192c04c040 con 0x55dee9bbfaf0 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1940dec740 10 client.16200 fetch_fsmap finished waiting for FSMap version 29 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1940dec740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1940dec740 20 client.16200 populate_metadata read hostname 'smithi092' 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1940dec740 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55dee9c3bb30 0x55dee9c3df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1940dec740 4 client.16200 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-19T02:03:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.392+0000 7f1940dec740 1 -- 172.21.15.92:0/471599209 --> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dee9b2b470 con 0x55dee9c3bb30 2024-03-19T02:03:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.393+0000 7f19377fe640 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55dee9c3bb30 0x55dee9c3df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.393+0000 7f1916ffd640 10 client.16200 ms_handle_connect on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:38.399 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.393+0000 7f19377fe640 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55dee9c3bb30 0x55dee9c3df50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b17c0ea7c9620ba server_cookie=716462c8a592e403 in_seq=0 out_seq=0 2024-03-19T02:03:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.399+0000 7f1916ffd640 1 -- 172.21.15.92:0/471599209 <== mds.0 v2:172.21.15.159:6838/2741545135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dee9b2b470 con 0x55dee9c3bb30 2024-03-19T02:03:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.399+0000 7f1916ffd640 10 client.16200 handle_command_reply: tid=0 2024-03-19T02:03:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.399+0000 7f1940dec740 1 client.16200 shutdown 2024-03-19T02:03:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.399+0000 7f1940dec740 1 -- 172.21.15.92:0/471599209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbfaf0 msgr2=0x55dee9bce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.399+0000 7f1940dec740 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbfaf0 0x55dee9bce270 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f192c036040 tx=0x7f192c005fd0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1940dec740 1 -- 172.21.15.92:0/471599209 shutdown_connections 2024-03-19T02:03:38.400 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1940dec740 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.159:6838/2741545135,v1:172.21.15.159:6839/2741545135] conn(0x55dee9c3bb30 0x55dee9c3df50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1940dec740 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dee9bc4c60 0x55dee9bcf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1916ffd640 0 client.16200 ms_handle_reset on v2:172.21.15.159:6838/2741545135 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1940dec740 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dee9bbfaf0 0x55dee9bce270 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1940dec740 1 --2- 172.21.15.92:0/471599209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dee9bbf100 0x55dee9bcdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1940dec740 1 -- 172.21.15.92:0/471599209 >> 172.21.15.92:0/471599209 conn(0x55dee9bba670 msgr2=0x55dee9bc01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1940dec740 1 -- 172.21.15.92:0/471599209 shutdown_connections 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1940dec740 1 -- 172.21.15.92:0/471599209 wait complete. 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f1940dec740 20 client.16200 trim_cache size 0 max 16384 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:{ 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "message": "", 2024-03-19T02:03:38.401 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout: "return_code": 0 2024-03-19T02:03:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stdout:} 2024-03-19T02:03:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 -- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f19200a28a0 msgr2=0x7f19200a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f19200a28a0 0x7f19200a4d60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f192c02c7b0 tx=0x7f192c02cdc0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 -- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19381022f0 msgr2=0x7f193819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19381022f0 0x7f193819f060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f193000eb10 tx=0x7f193000c2d0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 -- 172.21.15.92:0/3091967893 shutdown_connections 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19381022f0 0x7f193819f060 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1938101930 0x7f1938197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f19200a28a0 0x7f19200a4d60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 --2- 172.21.15.92:0/3091967893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1938100740 0x7f1938197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 -- 172.21.15.92:0/3091967893 >> 172.21.15.92:0/3091967893 conn(0x7f19380fbcb0 msgr2=0x7f19380fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 -- 172.21.15.92:0/3091967893 shutdown_connections 2024-03-19T02:03:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi092.front.sepia.ceph.com.smithi092.stderr:2024-03-19T02:03:38.400+0000 7f193e122640 1 -- 172.21.15.92:0/3091967893 wait complete. 2024-03-19T02:03:38.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:38 smithi159 ceph-mon[67110]: pgmap v676: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:03:39.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:38 smithi092 ceph-mon[62574]: pgmap v676: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:03:39.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:38 smithi144 ceph-mon[67199]: pgmap v676: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-03-19T02:03:40.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:40 smithi159 ceph-mon[67110]: pgmap v677: 97 pgs: 97 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-19T02:03:41.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:40 smithi092 ceph-mon[62574]: pgmap v677: 97 pgs: 97 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-19T02:03:41.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:40 smithi144 ceph-mon[67199]: pgmap v677: 97 pgs: 97 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:561/600: open fd = 3 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:562/600: open fd = 3 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:563/600: open fd = 3 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:564/600: open fd = 3 2024-03-19T02:03:42.619 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:565/600: open fd = 3 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:566/600: open fd = 3 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:567/600: open fd = 3 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.620 INFO:tasks.workunit.client.0.smithi092.stdout:568/600: open fd = 3 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:569/600: open fd = 3 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:570/600: open fd = 3 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:571/600: open fd = 3 2024-03-19T02:03:42.621 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:572/600: open fd = 3 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:573/600: open fd = 3 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:574/600: open fd = 3 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.622 INFO:tasks.workunit.client.0.smithi092.stdout:575/600: open fd = 3 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:576/600: open fd = 3 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:577/600: open fd = 3 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:578/600: open fd = 3 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.623 INFO:tasks.workunit.client.0.smithi092.stdout:579/600: open fd = 3 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:580/600: open fd = 3 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:581/600: open fd = 3 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:582/600: open fd = 3 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.624 INFO:tasks.workunit.client.0.smithi092.stdout:583/600: open fd = 3 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:584/600: open fd = 3 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:585/600: open fd = 3 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:586/600: open fd = 3 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:587/600: open fd = 3 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.625 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:588/600: open fd = 3 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:589/600: open fd = 3 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:590/600: open fd = 3 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.626 INFO:tasks.workunit.client.0.smithi092.stdout:591/600: open fd = 3 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:592/600: open fd = 3 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:593/600: open fd = 3 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:594/600: open fd = 3 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.627 INFO:tasks.workunit.client.0.smithi092.stdout:595/600: open fd = 3 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:596/600: open fd = 3 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:597/600: open fd = 3 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:598/600: open fd = 3 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.628 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.629 INFO:tasks.workunit.client.0.smithi092.stdout:599/600: open fd = 3 2024-03-19T02:03:42.629 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.629 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.629 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.629 INFO:tasks.workunit.client.0.smithi092.stdout:600/600: open fd = 3 2024-03-19T02:03:42.629 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.629 INFO:tasks.workunit.client.0.smithi092.stdout:write ret = 32 2024-03-19T02:03:42.629 INFO:tasks.workunit.client.0.smithi092.stdout:pread ret = 64 2024-03-19T02:03:42.629 INFO:tasks.workunit.client.0.smithi092.stderr:+ echo OK 2024-03-19T02:03:42.630 INFO:tasks.workunit.client.0.smithi092.stdout:OK 2024-03-19T02:03:42.631 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-19T02:03:42.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-19T02:03:42.722 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-03-19T02:03:42.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-19T02:03:42.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:42 smithi159 ceph-mon[67110]: pgmap v678: 97 pgs: 97 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T02:03:43.000 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:42 smithi092 ceph-mon[62574]: pgmap v678: 97 pgs: 97 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T02:03:43.001 DEBUG:teuthology.parallel:result is None 2024-03-19T02:03:43.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-19T02:03:43.038 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-19T02:03:43.038 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-19T02:03:43.049 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi092.front.sepia.ceph.com 2024-03-19T02:03:43.086 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-19T02:03:43.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T02:03:43.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:42 smithi144 ceph-mon[67199]: pgmap v678: 97 pgs: 97 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-19T02:03:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.228+0000 7febd8463640 1 -- 172.21.15.92:0/2050487186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0102360 msgr2=0x7febd0104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.228+0000 7febd8463640 1 --2- 172.21.15.92:0/2050487186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0102360 0x7febd0104750 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7febcc009900 tx=0x7febcc02bc30 comp rx=0 tx=0).stop 2024-03-19T02:03:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 -- 172.21.15.92:0/2050487186 shutdown_connections 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 --2- 172.21.15.92:0/2050487186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd01075c0 0x7febd01099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 --2- 172.21.15.92:0/2050487186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd0104c90 0x7febd0107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 --2- 172.21.15.92:0/2050487186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0102360 0x7febd0104750 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 -- 172.21.15.92:0/2050487186 >> 172.21.15.92:0/2050487186 conn(0x7febd00fbcb0 msgr2=0x7febd00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 -- 172.21.15.92:0/2050487186 shutdown_connections 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 -- 172.21.15.92:0/2050487186 wait complete. 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 Processor -- start 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 -- start start 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd0102360 0x7febd0101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0104c90 0x7febd0100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd01075c0 0x7febd0100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7febd010d4b0 con 0x7febd0104c90 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febd010d330 con 0x7febd0102360 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd8463640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7febd010d630 con 0x7febd01075c0 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd59fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0104c90 0x7febd0100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd59fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0104c90 0x7febd0100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42264/0 (socket says 172.21.15.92:42264) 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd59fe640 1 -- 172.21.15.92:0/1218699253 learned_addr learned my addr 172.21.15.92:0/1218699253 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd6a00640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd01075c0 0x7febd0100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.229+0000 7febd61ff640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd0102360 0x7febd0101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febd59fe640 1 -- 172.21.15.92:0/1218699253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd01075c0 msgr2=0x7febd0100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febd59fe640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd01075c0 0x7febd0100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febd59fe640 1 -- 172.21.15.92:0/1218699253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd0102360 msgr2=0x7febd0101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febd59fe640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd0102360 0x7febd0101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febd59fe640 1 -- 172.21.15.92:0/1218699253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd0100d90 con 0x7febd0104c90 2024-03-19T02:03:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febd59fe640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0104c90 0x7febd0100310 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7febc000e9c0 tx=0x7febc000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febc77fe640 1 -- 172.21.15.92:0/1218699253 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc000cd80 con 0x7febd0104c90 2024-03-19T02:03:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd0101080 con 0x7febd0104c90 2024-03-19T02:03:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febc77fe640 1 -- 172.21.15.92:0/1218699253 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc0005500 con 0x7febd0104c90 2024-03-19T02:03:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febc77fe640 1 -- 172.21.15.92:0/1218699253 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0010690 con 0x7febd0104c90 2024-03-19T02:03:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.230+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd01a6630 con 0x7febd0104c90 2024-03-19T02:03:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.231+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd0068610 con 0x7febd0104c90 2024-03-19T02:03:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.233+0000 7febc77fe640 1 -- 172.21.15.92:0/1218699253 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7febc00046c0 con 0x7febd0104c90 2024-03-19T02:03:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.233+0000 7febc77fe640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7febbc0a2800 0x7febbc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.234+0000 7febc77fe640 1 -- 172.21.15.92:0/1218699253 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7febc00cefc0 con 0x7febd0104c90 2024-03-19T02:03:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.234+0000 7febd61ff640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7febbc0a2800 0x7febbc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.234+0000 7febd61ff640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7febbc0a2800 0x7febbc0a4cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febcc02c7b0 tx=0x7febcc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:43.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.237+0000 7febc77fe640 1 -- 172.21.15.92:0/1218699253 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7febc0053740 con 0x7febd0104c90 2024-03-19T02:03:43.420 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.420+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febd00fd7a0 con 0x7febd0104c90 2024-03-19T02:03:43.422 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.422+0000 7febc77fe640 1 -- 172.21.15.92:0/1218699253 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22690 (secure 0 0 0) 0x7febd00fd7a0 con 0x7febd0104c90 2024-03-19T02:03:43.423 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T02:03:43.423 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":87,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:51:40.581546+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T02:03:43.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7febbc0a2800 msgr2=0x7febbc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7febbc0a2800 0x7febbc0a4cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febcc02c7b0 tx=0x7febcc005fd0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0104c90 msgr2=0x7febd0100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0104c90 0x7febd0100310 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7febc000e9c0 tx=0x7febc000ee80 comp rx=0 tx=0).stop 2024-03-19T02:03:43.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 shutdown_connections 2024-03-19T02:03:43.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7febbc0a2800 0x7febbc0a4cc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd01075c0 0x7febd0100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0104c90 0x7febd0100310 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.425 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 --2- 172.21.15.92:0/1218699253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd0102360 0x7febd0101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.424+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 >> 172.21.15.92:0/1218699253 conn(0x7febd00fbcb0 msgr2=0x7febd0102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:43.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.425+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 shutdown_connections 2024-03-19T02:03:43.426 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.425+0000 7febd8463640 1 -- 172.21.15.92:0/1218699253 wait complete. 2024-03-19T02:03:43.441 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:46:07.695753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:50:47.888441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:50:49.068308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T02:03:43.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T02:03:43.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.551+0000 7f77e5473640 1 -- 172.21.15.92:0/2337371073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e01075a0 msgr2=0x7f77e0109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.551+0000 7f77e5473640 1 --2- 172.21.15.92:0/2337371073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e01075a0 0x7f77e0109990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009ca0 tx=0x7f77d802c0f0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.551+0000 7f77e5473640 1 -- 172.21.15.92:0/2337371073 shutdown_connections 2024-03-19T02:03:43.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.551+0000 7f77e5473640 1 --2- 172.21.15.92:0/2337371073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e01075a0 0x7f77e0109990 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.554 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.551+0000 7f77e5473640 1 --2- 172.21.15.92:0/2337371073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e0104c70 0x7f77e0107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.551+0000 7f77e5473640 1 --2- 172.21.15.92:0/2337371073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e0102340 0x7f77e0104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.551+0000 7f77e5473640 1 -- 172.21.15.92:0/2337371073 >> 172.21.15.92:0/2337371073 conn(0x7f77e00fbcb0 msgr2=0x7f77e00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.551+0000 7f77e5473640 1 -- 172.21.15.92:0/2337371073 shutdown_connections 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.551+0000 7f77e5473640 1 -- 172.21.15.92:0/2337371073 wait complete. 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77e5473640 1 Processor -- start 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77e5473640 1 -- start start 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77e5473640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e0102340 0x7f77e0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77e5473640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e0104c70 0x7f77e0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77e5473640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e01075a0 0x7f77e019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77e5473640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f77e010d4a0 con 0x7f77e0104c70 2024-03-19T02:03:43.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77e5473640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77e010d320 con 0x7f77e0102340 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77e5473640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f77e010d620 con 0x7f77e01075a0 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77de7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e0104c70 0x7f77e0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77de7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e0104c70 0x7f77e0198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42284/0 (socket says 172.21.15.92:42284) 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77de7fc640 1 -- 172.21.15.92:0/364822246 learned_addr learned my addr 172.21.15.92:0/364822246 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77df7fe640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e01075a0 0x7f77e019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.552+0000 7f77deffd640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e0102340 0x7f77e0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77de7fc640 1 -- 172.21.15.92:0/364822246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e01075a0 msgr2=0x7f77e019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77de7fc640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e01075a0 0x7f77e019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77de7fc640 1 -- 172.21.15.92:0/364822246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e0102340 msgr2=0x7f77e0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77de7fc640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e0102340 0x7f77e0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.556 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77de7fc640 1 -- 172.21.15.92:0/364822246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e019f760 con 0x7f77e0104c70 2024-03-19T02:03:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77df7fe640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e01075a0 0x7f77e019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:03:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77deffd640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e0102340 0x7f77e0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:03:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77de7fc640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e0104c70 0x7f77e0198080 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f77d000e9c0 tx=0x7f77d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77c7fff640 1 -- 172.21.15.92:0/364822246 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d000cd80 con 0x7f77e0104c70 2024-03-19T02:03:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77c7fff640 1 -- 172.21.15.92:0/364822246 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0005500 con 0x7f77e0104c70 2024-03-19T02:03:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77c7fff640 1 -- 172.21.15.92:0/364822246 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0010690 con 0x7f77e0104c70 2024-03-19T02:03:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e019fa50 con 0x7f77e0104c70 2024-03-19T02:03:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.553+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e019fff0 con 0x7f77e0104c70 2024-03-19T02:03:43.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.554+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e00fd7a0 con 0x7f77e0104c70 2024-03-19T02:03:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.555+0000 7f77c7fff640 1 -- 172.21.15.92:0/364822246 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f77d00046c0 con 0x7f77e0104c70 2024-03-19T02:03:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.556+0000 7f77c7fff640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f77c80a2880 0x7f77c80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.556+0000 7f77deffd640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f77c80a2880 0x7f77c80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.556+0000 7f77c7fff640 1 -- 172.21.15.92:0/364822246 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f77d00ced30 con 0x7f77e0104c70 2024-03-19T02:03:43.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.556+0000 7f77deffd640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f77c80a2880 0x7f77c80a4d40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f77d40095f0 tx=0x7f77d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:43.560 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.560+0000 7f77c7fff640 1 -- 172.21.15.92:0/364822246 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f77d009bde0 con 0x7f77e0104c70 2024-03-19T02:03:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.780+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77e0062420 con 0x7f77e0104c70 2024-03-19T02:03:43.780 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.780+0000 7f77c7fff640 1 -- 172.21.15.92:0/364822246 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f77d009b6e0 con 0x7f77e0104c70 2024-03-19T02:03:43.781 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-03-19T02:03:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f77c80a2880 msgr2=0x7f77c80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f77c80a2880 0x7f77c80a4d40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f77d40095f0 tx=0x7f77d4005f90 comp rx=0 tx=0).stop 2024-03-19T02:03:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e0104c70 msgr2=0x7f77e0198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e0104c70 0x7f77e0198080 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f77d000e9c0 tx=0x7f77d000ee80 comp rx=0 tx=0).stop 2024-03-19T02:03:43.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 shutdown_connections 2024-03-19T02:03:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f77c80a2880 0x7f77c80a4d40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e01075a0 0x7f77e019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77e0104c70 0x7f77e0198080 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 --2- 172.21.15.92:0/364822246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e0102340 0x7f77e0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.782+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 >> 172.21.15.92:0/364822246 conn(0x7f77e00fbcb0 msgr2=0x7f77e0102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.783+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 shutdown_connections 2024-03-19T02:03:43.784 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.783+0000 7f77e5473640 1 -- 172.21.15.92:0/364822246 wait complete. 2024-03-19T02:03:43.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T02:03:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 -- 172.21.15.92:0/851901034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 msgr2=0x7fcf88100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 --2- 172.21.15.92:0/851901034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 0x7fcf88100b40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009900 tx=0x7fcf7802bc30 comp rx=0 tx=0).stop 2024-03-19T02:03:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 -- 172.21.15.92:0/851901034 shutdown_connections 2024-03-19T02:03:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 --2- 172.21.15.92:0/851901034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf881022d0 0x7fcf88108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 --2- 172.21.15.92:0/851901034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf88101910 0x7fcf88101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 --2- 172.21.15.92:0/851901034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 0x7fcf88100b40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 -- 172.21.15.92:0/851901034 >> 172.21.15.92:0/851901034 conn(0x7fcf880fbcb0 msgr2=0x7fcf880fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 -- 172.21.15.92:0/851901034 shutdown_connections 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 -- 172.21.15.92:0/851901034 wait complete. 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.900+0000 7fcf8de05640 1 Processor -- start 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf8de05640 1 -- start start 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf8de05640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 0x7fcf88197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf8de05640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf88101910 0x7fcf88198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf8de05640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf881022d0 0x7fcf8819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf8de05640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8810c3a0 con 0x7fcf88100720 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf8de05640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8810c220 con 0x7fcf88101910 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf8de05640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8810c520 con 0x7fcf881022d0 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf877fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 0x7fcf88197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf877fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 0x7fcf88197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58978/0 (socket says 172.21.15.92:58978) 2024-03-19T02:03:43.904 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf877fe640 1 -- 172.21.15.92:0/1493311694 learned_addr learned my addr 172.21.15.92:0/1493311694 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf87fff640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf881022d0 0x7fcf8819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf86ffd640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf88101910 0x7fcf88198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf877fe640 1 -- 172.21.15.92:0/1493311694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf881022d0 msgr2=0x7fcf8819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf877fe640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf881022d0 0x7fcf8819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf877fe640 1 -- 172.21.15.92:0/1493311694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf88101910 msgr2=0x7fcf88198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf877fe640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf88101910 0x7fcf88198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf877fe640 1 -- 172.21.15.92:0/1493311694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8819f760 con 0x7fcf88100720 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.901+0000 7fcf877fe640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 0x7fcf88197b40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcf78000c00 tx=0x7fcf7802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.902+0000 7fcf84ff9640 1 -- 172.21.15.92:0/1493311694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78041070 con 0x7fcf88100720 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.902+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8819f9f0 con 0x7fcf88100720 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.902+0000 7fcf84ff9640 1 -- 172.21.15.92:0/1493311694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf78004450 con 0x7fcf88100720 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.902+0000 7fcf84ff9640 1 -- 172.21.15.92:0/1493311694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78034440 con 0x7fcf88100720 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.902+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8819fe90 con 0x7fcf88100720 2024-03-19T02:03:43.905 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.903+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf881a0060 con 0x7fcf88100720 2024-03-19T02:03:43.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.904+0000 7fcf84ff9640 1 -- 172.21.15.92:0/1493311694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fcf78044070 con 0x7fcf88100720 2024-03-19T02:03:43.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.904+0000 7fcf84ff9640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf700a2800 0x7fcf700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:43.906 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.904+0000 7fcf86ffd640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf700a2800 0x7fcf700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:43.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.904+0000 7fcf84ff9640 1 -- 172.21.15.92:0/1493311694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcf780f0740 con 0x7fcf88100720 2024-03-19T02:03:43.907 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.905+0000 7fcf86ffd640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf700a2800 0x7fcf700a4cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcf88199050 tx=0x7fcf7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:43.908 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:43.909+0000 7fcf84ff9640 1 -- 172.21.15.92:0/1493311694 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcf780bd230 con 0x7fcf88100720 2024-03-19T02:03:43.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:43 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1218699253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T02:03:44.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:43 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1218699253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T02:03:44.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.090+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf881a0230 con 0x7fcf88100720 2024-03-19T02:03:44.090 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.090+0000 7fcf84ff9640 1 -- 172.21.15.92:0/1493311694 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fcf780c12d0 con 0x7fcf88100720 2024-03-19T02:03:44.091 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-03-19T02:03:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf700a2800 msgr2=0x7fcf700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf700a2800 0x7fcf700a4cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcf88199050 tx=0x7fcf7c005f90 comp rx=0 tx=0).stop 2024-03-19T02:03:44.093 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 msgr2=0x7fcf88197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 0x7fcf88197b40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fcf78000c00 tx=0x7fcf7802c640 comp rx=0 tx=0).stop 2024-03-19T02:03:44.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 shutdown_connections 2024-03-19T02:03:44.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fcf700a2800 0x7fcf700a4cc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf881022d0 0x7fcf8819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf88101910 0x7fcf88198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 --2- 172.21.15.92:0/1493311694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf88100720 0x7fcf88197b40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 >> 172.21.15.92:0/1493311694 conn(0x7fcf880fbcb0 msgr2=0x7fcf88068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:44.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 shutdown_connections 2024-03-19T02:03:44.094 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.093+0000 7fcf8de05640 1 -- 172.21.15.92:0/1493311694 wait complete. 2024-03-19T02:03:44.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:43 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1218699253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T02:03:44.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T02:03:44.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.210+0000 7f3958697640 1 -- 172.21.15.92:0/4005150068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3950101910 msgr2=0x7f3950101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.210+0000 7f3958697640 1 --2- 172.21.15.92:0/4005150068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3950101910 0x7f3950101d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3940009860 tx=0x7f394002bb70 comp rx=0 tx=0).stop 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.210+0000 7f3958697640 1 -- 172.21.15.92:0/4005150068 shutdown_connections 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.210+0000 7f3958697640 1 --2- 172.21.15.92:0/4005150068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39501022d0 0x7f3950108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.210+0000 7f3958697640 1 --2- 172.21.15.92:0/4005150068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3950101910 0x7f3950101d90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.210+0000 7f3958697640 1 --2- 172.21.15.92:0/4005150068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3950100720 0x7f3950100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.210+0000 7f3958697640 1 -- 172.21.15.92:0/4005150068 >> 172.21.15.92:0/4005150068 conn(0x7f39500fbcb0 msgr2=0x7f39500fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.210+0000 7f3958697640 1 -- 172.21.15.92:0/4005150068 shutdown_connections 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.210+0000 7f3958697640 1 -- 172.21.15.92:0/4005150068 wait complete. 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3958697640 1 Processor -- start 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3958697640 1 -- start start 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3958697640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3950100720 0x7f39501046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:44.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3958697640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3950101910 0x7f3950102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3958697640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39501022d0 0x7f3950103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3958697640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f395010c3a0 con 0x7f3950101910 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3958697640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f395010c220 con 0x7f3950100720 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3958697640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f395010c520 con 0x7f39501022d0 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3955c32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3950101910 0x7f3950102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3955c32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3950101910 0x7f3950102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59000/0 (socket says 172.21.15.92:59000) 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3955c32640 1 -- 172.21.15.92:0/2655405219 learned_addr learned my addr 172.21.15.92:0/2655405219 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3956433640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3950100720 0x7f39501046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3956c34640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39501022d0 0x7f3950103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3955c32640 1 -- 172.21.15.92:0/2655405219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39501022d0 msgr2=0x7f3950103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3955c32640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39501022d0 0x7f3950103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3955c32640 1 -- 172.21.15.92:0/2655405219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3950100720 msgr2=0x7f39501046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3955c32640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3950100720 0x7f39501046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3955c32640 1 -- 172.21.15.92:0/2655405219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3950103ac0 con 0x7f3950101910 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3956433640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3950100720 0x7f39501046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.211+0000 7f3955c32640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3950101910 0x7f3950102cf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3940005940 tx=0x7f3940032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.212+0000 7f39477fe640 1 -- 172.21.15.92:0/2655405219 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940041020 con 0x7f3950101910 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.212+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39501a1c50 con 0x7f3950101910 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.212+0000 7f39477fe640 1 -- 172.21.15.92:0/2655405219 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f394002c9f0 con 0x7f3950101910 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.212+0000 7f39477fe640 1 -- 172.21.15.92:0/2655405219 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940039040 con 0x7f3950101910 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.212+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39501a21a0 con 0x7f3950101910 2024-03-19T02:03:44.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.213+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39501a2370 con 0x7f3950101910 2024-03-19T02:03:44.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.214+0000 7f39477fe640 1 -- 172.21.15.92:0/2655405219 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f3940044020 con 0x7f3950101910 2024-03-19T02:03:44.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.214+0000 7f39477fe640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f393c0a2800 0x7f393c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:44.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.215+0000 7f3956433640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f393c0a2800 0x7f393c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:44.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.215+0000 7f39477fe640 1 -- 172.21.15.92:0/2655405219 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f39400f0a20 con 0x7f3950101910 2024-03-19T02:03:44.216 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.215+0000 7f3956433640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f393c0a2800 0x7f393c0a4cc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3950101770 tx=0x7f394c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:44.218 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.218+0000 7f39477fe640 1 -- 172.21.15.92:0/2655405219 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f394004b020 con 0x7f3950101910 2024-03-19T02:03:44.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.394+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f39501a2700 con 0x7f3950101910 2024-03-19T02:03:44.395 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.395+0000 7f39477fe640 1 -- 172.21.15.92:0/2655405219 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f39400bdbf0 con 0x7f3950101910 2024-03-19T02:03:44.395 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-03-19T02:03:44.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f393c0a2800 msgr2=0x7f393c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f393c0a2800 0x7f393c0a4cc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3950101770 tx=0x7f394c009260 comp rx=0 tx=0).stop 2024-03-19T02:03:44.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3950101910 msgr2=0x7f3950102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3950101910 0x7f3950102cf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3940005940 tx=0x7f3940032470 comp rx=0 tx=0).stop 2024-03-19T02:03:44.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 shutdown_connections 2024-03-19T02:03:44.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f393c0a2800 0x7f393c0a4cc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39501022d0 0x7f3950103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3950101910 0x7f3950102cf0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 --2- 172.21.15.92:0/2655405219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3950100720 0x7f39501046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 >> 172.21.15.92:0/2655405219 conn(0x7f39500fbcb0 msgr2=0x7f39500fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:44.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 shutdown_connections 2024-03-19T02:03:44.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.397+0000 7f3958697640 1 -- 172.21.15.92:0/2655405219 wait complete. 2024-03-19T02:03:44.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-19T02:03:44.524 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.521+0000 7f6fee8ac640 1 -- 172.21.15.92:0/4040406034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8101860 msgr2=0x7f6fe8101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.521+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/4040406034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8101860 0x7f6fe8101c80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009900 tx=0x7f6fdc02bc30 comp rx=0 tx=0).stop 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.521+0000 7f6fee8ac640 1 -- 172.21.15.92:0/4040406034 shutdown_connections 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.521+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/4040406034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fe8103410 0x7f6fe8109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.521+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/4040406034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe8102a50 0x7f6fe8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.521+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/4040406034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8101860 0x7f6fe8101c80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 -- 172.21.15.92:0/4040406034 >> 172.21.15.92:0/4040406034 conn(0x7f6fe80fcdd0 msgr2=0x7f6fe80ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 -- 172.21.15.92:0/4040406034 shutdown_connections 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 -- 172.21.15.92:0/4040406034 wait complete. 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 Processor -- start 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 -- start start 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe8101860 0x7f6fe8197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:44.525 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fe8102a50 0x7f6fe8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8103410 0x7f6fe819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe810d4b0 con 0x7f6fe8103410 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe810d330 con 0x7f6fe8101860 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.522+0000 7f6fee8ac640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe810d630 con 0x7f6fe8102a50 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fece49640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8103410 0x7f6fe819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fece49640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8103410 0x7f6fe819f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59016/0 (socket says 172.21.15.92:59016) 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fece49640 1 -- 172.21.15.92:0/63654038 learned_addr learned my addr 172.21.15.92:0/63654038 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fe77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fe8102a50 0x7f6fe8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fe7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe8101860 0x7f6fe8197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fece49640 1 -- 172.21.15.92:0/63654038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fe8102a50 msgr2=0x7f6fe8198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fece49640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fe8102a50 0x7f6fe8198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fece49640 1 -- 172.21.15.92:0/63654038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe8101860 msgr2=0x7f6fe8197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:44.526 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fece49640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe8101860 0x7f6fe8197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:44.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fece49640 1 -- 172.21.15.92:0/63654038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe819f7f0 con 0x7f6fe8103410 2024-03-19T02:03:44.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fe77fe640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fe8102a50 0x7f6fe8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:03:44.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fece49640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8103410 0x7f6fe819f180 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000efd0 tx=0x7f6fe000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:44.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fe57fa640 1 -- 172.21.15.92:0/63654038 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0003b70 con 0x7f6fe8103410 2024-03-19T02:03:44.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe819fae0 con 0x7f6fe8103410 2024-03-19T02:03:44.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fe57fa640 1 -- 172.21.15.92:0/63654038 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fe0003d10 con 0x7f6fe8103410 2024-03-19T02:03:44.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe81a0030 con 0x7f6fe8103410 2024-03-19T02:03:44.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.523+0000 7f6fe57fa640 1 -- 172.21.15.92:0/63654038 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00109f0 con 0x7f6fe8103410 2024-03-19T02:03:44.527 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.525+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe8108c50 con 0x7f6fe8103410 2024-03-19T02:03:44.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.525+0000 7f6fe57fa640 1 -- 172.21.15.92:0/63654038 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f6fe0004020 con 0x7f6fe8103410 2024-03-19T02:03:44.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.526+0000 7f6fe57fa640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6fd00a2800 0x7f6fd00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:03:44.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.526+0000 7f6fe7fff640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6fd00a2800 0x7f6fd00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:03:44.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.526+0000 7f6fe57fa640 1 -- 172.21.15.92:0/63654038 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6fe00cee80 con 0x7f6fe8103410 2024-03-19T02:03:44.528 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.526+0000 7f6fe7fff640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6fd00a2800 0x7f6fd00a4cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc000c00 tx=0x7f6fdc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:03:44.530 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.530+0000 7f6fe57fa640 1 -- 172.21.15.92:0/63654038 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6fe009bf30 con 0x7f6fe8103410 2024-03-19T02:03:44.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.761+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f6fe8062420 con 0x7f6fe8103410 2024-03-19T02:03:44.769 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.769+0000 7f6fe57fa640 1 -- 172.21.15.92:0/63654038 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+739830 (secure 0 0 0) 0x7f6fe009b830 con 0x7f6fe8103410 2024-03-19T02:03:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T02:03:44.781 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T02:03:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 20, 2024-03-19T02:03:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "active_gid": 14152, 2024-03-19T02:03:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "active_name": "x", 2024-03-19T02:03:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "active_addrs": { 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6800", 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3230736370 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6801", 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3230736370 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "active_addr": "172.21.15.92:6801/3230736370", 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "active_change": "2024-03-19T01:43:39.363264+0000", 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "active_mgr_features": 4540701547738038271, 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "available": true, 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "standbys": [ 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "gid": 24109, 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "z", 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_features": 4540701547738038271, 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "available_modules": [ 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "alerts", 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How frequently to reexamine health status", 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_destination": { 2024-03-19T02:03:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_destination", 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email address to send alerts to", 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_from_name": { 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_from_name", 2024-03-19T02:03:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "Ceph", 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email From: name", 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_host": { 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_host", 2024-03-19T02:03:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP server", 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_password": { 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_password", 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Password to authenticate with", 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_port": { 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_port", 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "465", 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP port", 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_sender": { 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_sender", 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP envelope sender", 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_ssl": { 2024-03-19T02:03:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_ssl", 2024-03-19T02:03:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_user": { 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_user", 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "User to authenticate as", 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "balancer", 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "active": { 2024-03-19T02:03:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "active", 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically balance PGs across cluster", 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_time": { 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_time", 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0000", 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "beginning time of day to automatically balance", 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_weekday": { 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_weekday", 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_max_iterations": { 2024-03-19T02:03:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_max_iterations", 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "25", 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "250", 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_metrics": { 2024-03-19T02:03:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_metrics", 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "pgs,objects,bytes", 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_step": { 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_step", 2024-03-19T02:03:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.5", 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0.001", 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "0.999", 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "aggressiveness of optimization", 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "end_time": { 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_time", 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2359", 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ending time of day to automatically balance", 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "end_weekday": { 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_weekday", 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "min_score": { 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_score", 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "upmap", 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-compat", 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "none", 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "read", 2024-03-19T02:03:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap", 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap-read" 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Balancer mode", 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_ids": { 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_ids", 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_deviation": { 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_deviation", 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_optimizations": { 2024-03-19T02:03:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_optimizations", 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm", 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_down_multiplier": { 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_down_multiplier", 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_refresh_rate": { 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_refresh_rate", 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "20", 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_starting_port": { 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_starting_port", 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4721", 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_ptrace": { 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_ptrace", 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_interval": { 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_interval", 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_memory_target_ratio": { 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_memory_target_ratio", 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.7", 2024-03-19T02:03:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-19T02:03:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_log_destination": { 2024-03-19T02:03:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm_log_destination", 2024-03-19T02:03:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "file", 2024-03-19T02:03:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "file,syslog", 2024-03-19T02:03:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "syslog" 2024-03-19T02:03:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-19T02:03:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "cgroups_split": { 2024-03-19T02:03:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cgroups_split", 2024-03-19T02:03:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-19T02:03:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "config_checks_enabled": { 2024-03-19T02:03:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_checks_enabled", 2024-03-19T02:03:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-19T02:03:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "config_dashboard": { 2024-03-19T02:03:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_dashboard", 2024-03-19T02:03:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-19T02:03:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_alertmanager": { 2024-03-19T02:03:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_alertmanager", 2024-03-19T02:03:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-19T02:03:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_base": { 2024-03-19T02:03:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_base", 2024-03-19T02:03:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-19T02:03:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Container image name, without the tag", 2024-03-19T02:03:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_elasticsearch": { 2024-03-19T02:03:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_elasticsearch", 2024-03-19T02:03:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-19T02:03:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "elasticsearch container image", 2024-03-19T02:03:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_grafana": { 2024-03-19T02:03:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_grafana", 2024-03-19T02:03:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-19T02:03:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_haproxy": { 2024-03-19T02:03:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_haproxy", 2024-03-19T02:03:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-19T02:03:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "HAproxy container image", 2024-03-19T02:03:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_agent": { 2024-03-19T02:03:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_agent", 2024-03-19T02:03:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-19T02:03:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger agent container image", 2024-03-19T02:03:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_collector": { 2024-03-19T02:03:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_collector", 2024-03-19T02:03:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-19T02:03:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger collector container image", 2024-03-19T02:03:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_query": { 2024-03-19T02:03:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_query", 2024-03-19T02:03:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-19T02:03:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger query container image", 2024-03-19T02:03:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_keepalived": { 2024-03-19T02:03:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_keepalived", 2024-03-19T02:03:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-19T02:03:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Keepalived container image", 2024-03-19T02:03:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_loki": { 2024-03-19T02:03:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_loki", 2024-03-19T02:03:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-19T02:03:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Loki container image", 2024-03-19T02:03:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_node_exporter": { 2024-03-19T02:03:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_node_exporter", 2024-03-19T02:03:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-19T02:03:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.859 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nvmeof": { 2024-03-19T02:03:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nvmeof", 2024-03-19T02:03:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.861 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-19T02:03:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nvme-of container image", 2024-03-19T02:03:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.862 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_prometheus": { 2024-03-19T02:03:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_prometheus", 2024-03-19T02:03:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-19T02:03:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:44.864 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_promtail": { 2024-03-19T02:03:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_promtail", 2024-03-19T02:03:44.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-19T02:03:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.866 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Promtail container image", 2024-03-19T02:03:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_snmp_gateway": { 2024-03-19T02:03:44.867 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_snmp_gateway", 2024-03-19T02:03:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-19T02:03:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.868 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SNMP Gateway container image", 2024-03-19T02:03:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "container_init": { 2024-03-19T02:03:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_init", 2024-03-19T02:03:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Run podman/docker with `--init`", 2024-03-19T02:03:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.870 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "daemon_cache_timeout": { 2024-03-19T02:03:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "daemon_cache_timeout", 2024-03-19T02:03:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:44.871 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "default_cephadm_command_timeout": { 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_cephadm_command_timeout", 2024-03-19T02:03:44.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "900", 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "default_registry": { 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_registry", 2024-03-19T02:03:44.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io", 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "device_cache_timeout": { 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_cache_timeout", 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1800", 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache device inventory", 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "device_enhanced_scan": { 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_enhanced_scan", 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.876 INFO:teuthology.orchestra.run.smithi092.stdout: "facts_cache_timeout": { 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "facts_cache_timeout", 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache host facts data", 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "host_check_interval": { 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "host_check_interval", 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to perform a host check", 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.878 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "hw_monitoring": { 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hw_monitoring", 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "inventory_list_all": { 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "inventory_list_all", 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.881 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "log_refresh_metadata": { 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_refresh_metadata", 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.882 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.883 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.885 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.886 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.888 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.889 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-19T02:03:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-19T02:03:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-19T02:03:44.891 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-19T02:03:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-19T02:03:44.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "*", 2024-03-19T02:03:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.893 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-19T02:03:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.894 INFO:teuthology.orchestra.run.smithi092.stdout: "max_count_per_host": { 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_count_per_host", 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of daemons per service per host", 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.895 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd_draining_count": { 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_osd_draining_count", 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "migration_current": { 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "migration_current", 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "internal - do not modify", 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "root", 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm-package", 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "root" 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "mode for remote execution of cephadm", 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "oob_default_addr": { 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "oob_default_addr", 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "169.254.1.1", 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.899 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "prometheus_alerts_path": { 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus_alerts_path", 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.900 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_insecure": { 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_insecure", 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_password": { 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_password", 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.902 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_url": { 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_url", 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_username": { 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_username", 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.904 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "secure_monitoring_stack": { 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secure_monitoring_stack", 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-19T02:03:44.905 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "service_discovery_port": { 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "service_discovery_port", 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8765", 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.906 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "cephadm service discovery port", 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_config_file": { 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_config_file", 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.907 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-19T02:03:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.908 INFO:teuthology.orchestra.run.smithi092.stdout: "use_agent": { 2024-03-19T02:03:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_agent", 2024-03-19T02:03:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.909 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-19T02:03:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.910 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "use_repo_digest": { 2024-03-19T02:03:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_repo_digest", 2024-03-19T02:03:44.911 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.912 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-19T02:03:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.913 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_failed_host_check": { 2024-03-19T02:03:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_failed_host_check", 2024-03-19T02:03:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.914 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if the host check fails", 2024-03-19T02:03:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.915 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_daemons": { 2024-03-19T02:03:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_daemons", 2024-03-19T02:03:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.916 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_hosts": { 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_hosts", 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.917 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crash", 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:44.918 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.919 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.920 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.921 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.922 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "retain_interval": { 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retain_interval", 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "31536000", 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_recent_interval": { 2024-03-19T02:03:44.924 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_recent_interval", 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1209600", 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dashboard", 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.926 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_HOST": { 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_ENABLED": { 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_ENABLED", 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.929 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.930 INFO:teuthology.orchestra.run.smithi092.stdout: "ENABLE_BROWSABLE_API": { 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.931 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-19T02:03:44.932 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-19T02:03:44.933 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-19T02:03:44.934 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-19T02:03:44.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.936 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-19T02:03:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-19T02:03:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.937 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.939 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.940 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_PASSWORD": { 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.941 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.942 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_URL": { 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_URL", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_USERNAME": { 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.944 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-19T02:03:44.946 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "MULTICLUSTER_CONFIG": { 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "{}", 2024-03-19T02:03:44.949 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_HOST": { 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.950 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.951 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.952 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.953 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.954 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.955 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.957 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-19T02:03:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-19T02:03:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.958 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.959 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_ENABLED": { 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.960 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-19T02:03:44.962 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-19T02:03:44.963 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8", 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-19T02:03:44.964 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "45", 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ACCESS_KEY": { 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-19T02:03:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SECRET_KEY": { 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SSL_VERIFY": { 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "UNSAFE_TLS_v1_2": { 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.972 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "cross_origin_url": { 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cross_origin_url", 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.973 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "crt_file": { 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crt_file", 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.974 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug": { 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "debug", 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.975 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable/disable debug options", 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "jwt_token_ttl": { 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "jwt_token_ttl", 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "28800", 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.976 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.977 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.978 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.979 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:44.980 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.981 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.982 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "motd": { 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "motd", 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.983 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "The message of the day", 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect_resolve_ip_addr": { 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "redirect_resolve_ip_addr", 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.984 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.985 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8080", 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.986 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-03-19T02:03:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-03-19T02:03:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.988 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl_server_port": { 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl_server_port", 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8443", 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "redirect", 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect" 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.990 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "url_prefix": { 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url_prefix", 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.992 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_monitoring": { 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_monitoring", 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.993 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "monitor device health metrics", 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.994 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.995 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.996 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:44.997 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:44.998 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "mark_out_threshold": { 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mark_out_threshold", 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2419200", 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": { 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_name", 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.000 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "device_health_metrics", 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "retention_period": { 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retention_period", 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15552000", 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain device health metrics", 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_frequency": { 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_frequency", 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "self_heal": { 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "self_heal", 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.004 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and check device health", 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_threshold": { 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_threshold", 2024-03-19T02:03:45.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7257600", 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.006 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "diskprediction_local", 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.007 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.008 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.009 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.010 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.011 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.012 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.013 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.014 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.015 INFO:teuthology.orchestra.run.smithi092.stdout: "predict_interval": { 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predict_interval", 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "predictor_model": { 2024-03-19T02:03:45.016 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predictor_model", 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "prophetstor", 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "influx", 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": false, 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "influxdb python module not found", 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "batch_size": { 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "batch_size", 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5000", 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-19T02:03:45.019 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "database": { 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "database", 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "ceph", 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-19T02:03:45.020 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": { 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hostname", 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server hostname", 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "30", 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "5", 2024-03-19T02:03:45.022 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.023 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.024 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.025 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.026 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.027 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "password": { 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "password", 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.028 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "password of InfluxDB server user", 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "port": { 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "port", 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8086", 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server port", 2024-03-19T02:03:45.029 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "false", 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.030 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "threads": { 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threads", 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "32", 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.031 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "username": { 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "username", 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "username of InfluxDB server user", 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.032 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "verify_ssl": { 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "verify_ssl", 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "true", 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.033 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "insights", 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.034 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.036 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.037 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.038 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "iostat", 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.039 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.040 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.041 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.043 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "k8sevents", 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_event_retention_days": { 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ceph_event_retention_days", 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-03-19T02:03:45.045 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "config_check_secs": { 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_check_secs", 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "10", 2024-03-19T02:03:45.046 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.049 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.050 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "localpool", 2024-03-19T02:03:45.052 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "failure_domain": { 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "failure_domain", 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "host", 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "failure domain for any created local pool", 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-19T02:03:45.053 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.054 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.055 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.056 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.057 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": { 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_size", 2024-03-19T02:03:45.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default min_size for any created local pool", 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "num_rep": { 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "num_rep", 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.059 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default replica count for any created local pool", 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": { 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_num", 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.060 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "128", 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default pg_num for any created local pool", 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "prefix": { 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prefix", 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name prefix for any created local pool", 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "subtree": { 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "subtree", 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.062 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "rack", 2024-03-19T02:03:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.063 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-19T02:03:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-19T02:03:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mds_autoscaler", 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.065 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.066 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.067 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.068 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.069 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.070 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.072 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.073 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.073 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.073 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirroring", 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.074 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.075 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.076 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.077 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.078 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.079 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "nfs", 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.080 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.081 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.082 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.083 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.084 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "fail_fs": { 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "fail_fs", 2024-03-19T02:03:45.085 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-19T02:03:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.087 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.088 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.089 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.090 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator": { 2024-03-19T02:03:45.091 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rook", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "test_orchestrator" 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Orchestrator backend", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_perf_query", 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.095 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.096 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.097 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.098 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_support", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.099 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.100 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.101 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.102 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_autoscaler", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.104 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.105 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.106 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.107 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.108 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.109 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "threshold": { 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threshold", 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1.0", 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "scaling threshold", 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "progress", 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_pg_recovery_event": { 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_pg_recovery_event", 2024-03-19T02:03:45.111 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-03-19T02:03:45.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.113 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.115 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.116 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "max_completed_events": { 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_completed_events", 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "50", 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "number of past completed events to remember", 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.119 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long the module is going to sleep", 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus", 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.120 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "cache": { 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cache", 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.121 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "exclude_perf_counters": { 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "exclude_perf_counters", 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.122 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-19T02:03:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-19T02:03:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.123 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.124 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.125 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.126 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.127 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.128 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.130 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools": { 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools", 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.132 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-19T02:03:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-19T02:03:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "300", 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_interval": { 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_interval", 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.134 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15.0", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.135 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "9283", 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.136 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "stale_cache_strategy": { 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stale_cache_strategy", 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "log", 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.137 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "default", 2024-03-19T02:03:45.138 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "default", 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "error" 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.139 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.140 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.141 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.142 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.145 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_snap_create": { 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_snap_create", 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.146 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "mirror_snapshot_schedule": { 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirror_snapshot_schedule", 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.147 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "trash_purge_schedule": { 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "trash_purge_schedule", 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.148 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "restful", 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_auth": { 2024-03-19T02:03:45.149 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_auth", 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.150 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.151 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.152 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.153 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.154 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.156 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.157 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rgw", 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.161 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.162 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.163 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rook", 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.165 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.166 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.167 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.168 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "storage_class": { 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "storage_class", 2024-03-19T02:03:45.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "local", 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "selftest", 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.171 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.172 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.173 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.174 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.176 INFO:teuthology.orchestra.run.smithi092.stdout: "roption1": { 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption1", 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "roption2": { 2024-03-19T02:03:45.177 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption2", 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "xyz", 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.178 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption1": { 2024-03-19T02:03:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption1", 2024-03-19T02:03:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.179 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption2": { 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption2", 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.181 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption3": { 2024-03-19T02:03:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption3", 2024-03-19T02:03:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption4": { 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption4", 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption5": { 2024-03-19T02:03:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption5", 2024-03-19T02:03:45.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption6": { 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption6", 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption7": { 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption7", 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "42", 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "testkey": { 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testkey", 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "testlkey": { 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testlkey", 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "testnewline": { 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testnewline", 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snap_schedule", 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_m_granularity": { 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_m_granularity", 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.193 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow minute scheduled snapshots", 2024-03-19T02:03:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.194 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "dump_on_update": { 2024-03-19T02:03:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dump_on_update", 2024-03-19T02:03:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.195 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.196 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "dump database to debug log on update", 2024-03-19T02:03:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.197 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.198 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.199 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.200 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.201 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.202 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.203 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.204 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stats", 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.205 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.206 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.207 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.208 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "status", 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.210 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.211 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.212 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.213 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.215 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telegraf", 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "address": { 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "address", 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.216 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15", 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.217 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.218 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.219 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.220 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.221 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.222 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.223 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telemetry", 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_basic": { 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_basic", 2024-03-19T02:03:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_crash": { 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_crash", 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.225 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_device": { 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_device", 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.226 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_ident": { 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_ident", 2024-03-19T02:03:45.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-19T02:03:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.228 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_perf": { 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_perf", 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "contact": { 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "contact", 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "description": { 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "description", 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "device_url": { 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_url", 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-03-19T02:03:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "24", 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "8", 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "last_opt_revision": { 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "last_opt_revision", 2024-03-19T02:03:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1", 2024-03-19T02:03:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard": { 2024-03-19T02:03:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard", 2024-03-19T02:03:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard_description": { 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard_description", 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "organization": { 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "organization", 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "proxy": { 2024-03-19T02:03:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "proxy", 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "url": { 2024-03-19T02:03:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url", 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "test_orchestrator", 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_clones": { 2024-03-19T02:03:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_clones", 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4", 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "periodic_async_work": { 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "periodic_async_work", 2024-03-19T02:03:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Periodically check for async work", 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_delay": { 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_delay", 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_no_wait": { 2024-03-19T02:03:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_no_wait", 2024-03-19T02:03:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-19T02:03:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix", 2024-03-19T02:03:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "discovery_interval": { 2024-03-19T02:03:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "discovery_interval", 2024-03-19T02:03:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "100", 2024-03-19T02:03:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "identifier": { 2024-03-19T02:03:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "identifier", 2024-03-19T02:03:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_host": { 2024-03-19T02:03:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_host", 2024-03-19T02:03:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_port": { 2024-03-19T02:03:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_port", 2024-03-19T02:03:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10051", 2024-03-19T02:03:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_sender": { 2024-03-19T02:03:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_sender", 2024-03-19T02:03:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-19T02:03:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T02:03:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "gid": 24119, 2024-03-19T02:03:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "y", 2024-03-19T02:03:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_features": 4540701547738038271, 2024-03-19T02:03:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "available_modules": [ 2024-03-19T02:03:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "alerts", 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How frequently to reexamine health status", 2024-03-19T02:03:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_destination": { 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_destination", 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email address to send alerts to", 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_from_name": { 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_from_name", 2024-03-19T02:03:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "Ceph", 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email From: name", 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_host": { 2024-03-19T02:03:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_host", 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP server", 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_password": { 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_password", 2024-03-19T02:03:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Password to authenticate with", 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_port": { 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_port", 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "465", 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP port", 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_sender": { 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_sender", 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP envelope sender", 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_ssl": { 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_ssl", 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_user": { 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_user", 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "User to authenticate as", 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "balancer", 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "active": { 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "active", 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically balance PGs across cluster", 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_time": { 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_time", 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0000", 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "beginning time of day to automatically balance", 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_weekday": { 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_weekday", 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_max_iterations": { 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_max_iterations", 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "25", 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "250", 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_metrics": { 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_metrics", 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "pgs,objects,bytes", 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_step": { 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_step", 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.5", 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0.001", 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "0.999", 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "aggressiveness of optimization", 2024-03-19T02:03:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "end_time": { 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_time", 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2359", 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ending time of day to automatically balance", 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T02:03:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "end_weekday": { 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_weekday", 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T02:03:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "min_score": { 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_score", 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "upmap", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-compat", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "none", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "read", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap", 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap-read" 2024-03-19T02:03:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Balancer mode", 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_ids": { 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_ids", 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_deviation": { 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_deviation", 2024-03-19T02:03:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_optimizations": { 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_optimizations", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm", 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_down_multiplier": { 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_down_multiplier", 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-19T02:03:45.331 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_refresh_rate": { 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_refresh_rate", 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "20", 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-19T02:03:45.332 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_starting_port": { 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_starting_port", 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4721", 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_ptrace": { 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_ptrace", 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.334 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_interval": { 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_interval", 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_memory_target_ratio": { 2024-03-19T02:03:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_memory_target_ratio", 2024-03-19T02:03:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.7", 2024-03-19T02:03:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.336 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_log_destination": { 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm_log_destination", 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.337 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "file", 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "file,syslog", 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "syslog" 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.338 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "cgroups_split": { 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cgroups_split", 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "config_checks_enabled": { 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_checks_enabled", 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.340 INFO:teuthology.orchestra.run.smithi092.stdout: "config_dashboard": { 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_dashboard", 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_alertmanager": { 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_alertmanager", 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.342 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_base": { 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_base", 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Container image name, without the tag", 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_elasticsearch": { 2024-03-19T02:03:45.343 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_elasticsearch", 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "elasticsearch container image", 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_grafana": { 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_grafana", 2024-03-19T02:03:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.345 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_haproxy": { 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_haproxy", 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "HAproxy container image", 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.346 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_agent": { 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_agent", 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.347 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger agent container image", 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_collector": { 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_collector", 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger collector container image", 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_query": { 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_query", 2024-03-19T02:03:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger query container image", 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_keepalived": { 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_keepalived", 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Keepalived container image", 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_loki": { 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_loki", 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Loki container image", 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_node_exporter": { 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_node_exporter", 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nvmeof": { 2024-03-19T02:03:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nvmeof", 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nvme-of container image", 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_prometheus": { 2024-03-19T02:03:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_prometheus", 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_promtail": { 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_promtail", 2024-03-19T02:03:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Promtail container image", 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_snmp_gateway": { 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_snmp_gateway", 2024-03-19T02:03:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SNMP Gateway container image", 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "container_init": { 2024-03-19T02:03:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_init", 2024-03-19T02:03:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Run podman/docker with `--init`", 2024-03-19T02:03:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "daemon_cache_timeout": { 2024-03-19T02:03:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "daemon_cache_timeout", 2024-03-19T02:03:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "default_cephadm_command_timeout": { 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_cephadm_command_timeout", 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "900", 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "default_registry": { 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_registry", 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io", 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "device_cache_timeout": { 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_cache_timeout", 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1800", 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache device inventory", 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "device_enhanced_scan": { 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_enhanced_scan", 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "facts_cache_timeout": { 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "facts_cache_timeout", 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache host facts data", 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "host_check_interval": { 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "host_check_interval", 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to perform a host check", 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "hw_monitoring": { 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hw_monitoring", 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-19T02:03:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "inventory_list_all": { 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "inventory_list_all", 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "log_refresh_metadata": { 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_refresh_metadata", 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-19T02:03:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "*", 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "max_count_per_host": { 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_count_per_host", 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of daemons per service per host", 2024-03-19T02:03:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd_draining_count": { 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_osd_draining_count", 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "migration_current": { 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "migration_current", 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "internal - do not modify", 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-03-19T02:03:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "root", 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm-package", 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "root" 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "mode for remote execution of cephadm", 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "oob_default_addr": { 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "oob_default_addr", 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "169.254.1.1", 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "prometheus_alerts_path": { 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus_alerts_path", 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_insecure": { 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_insecure", 2024-03-19T02:03:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_password": { 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_password", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_url": { 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_url", 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_username": { 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_username", 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "secure_monitoring_stack": { 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secure_monitoring_stack", 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "service_discovery_port": { 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "service_discovery_port", 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8765", 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "cephadm service discovery port", 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_config_file": { 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_config_file", 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "use_agent": { 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_agent", 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "use_repo_digest": { 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_repo_digest", 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_failed_host_check": { 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_failed_host_check", 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if the host check fails", 2024-03-19T02:03:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_daemons": { 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_daemons", 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_hosts": { 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_hosts", 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crash", 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "retain_interval": { 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retain_interval", 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "31536000", 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_recent_interval": { 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_recent_interval", 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1209600", 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dashboard", 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-19T02:03:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_HOST": { 2024-03-19T02:03:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-19T02:03:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_ENABLED": { 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_ENABLED", 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "ENABLE_BROWSABLE_API": { 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_PASSWORD": { 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_URL": { 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_URL", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_USERNAME": { 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "MULTICLUSTER_CONFIG": { 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "{}", 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_HOST": { 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_ENABLED": { 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8", 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "45", 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ACCESS_KEY": { 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-19T02:03:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SECRET_KEY": { 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SSL_VERIFY": { 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "UNSAFE_TLS_v1_2": { 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "cross_origin_url": { 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cross_origin_url", 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "crt_file": { 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crt_file", 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "debug": { 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "debug", 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable/disable debug options", 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "jwt_token_ttl": { 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "jwt_token_ttl", 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "28800", 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.455 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.456 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.457 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.458 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.459 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.460 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "motd": { 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "motd", 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.461 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "The message of the day", 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect_resolve_ip_addr": { 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "redirect_resolve_ip_addr", 2024-03-19T02:03:45.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8080", 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.465 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl_server_port": { 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl_server_port", 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8443", 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-03-19T02:03:45.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "redirect", 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect" 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.468 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.469 INFO:teuthology.orchestra.run.smithi092.stdout: "url_prefix": { 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url_prefix", 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_monitoring": { 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_monitoring", 2024-03-19T02:03:45.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "monitor device health metrics", 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.472 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.473 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.475 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.477 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mark_out_threshold": { 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mark_out_threshold", 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2419200", 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.478 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": { 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_name", 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "device_health_metrics", 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.479 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "retention_period": { 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retention_period", 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15552000", 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.480 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain device health metrics", 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_frequency": { 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_frequency", 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.481 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "self_heal": { 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "self_heal", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.482 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and check device health", 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.483 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_threshold": { 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_threshold", 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7257600", 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "diskprediction_local", 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.487 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.489 INFO:teuthology.orchestra.run.smithi092.stdout: "predict_interval": { 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predict_interval", 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "predictor_model": { 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predictor_model", 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "prophetstor", 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "influx", 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": false, 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "influxdb python module not found", 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "batch_size": { 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "batch_size", 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5000", 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "database": { 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "database", 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "ceph", 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": { 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hostname", 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.495 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server hostname", 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "30", 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "5", 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-19T02:03:45.496 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.497 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.498 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.499 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.500 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.501 INFO:teuthology.orchestra.run.smithi092.stdout: "password": { 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "password", 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "password of InfluxDB server user", 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.502 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "port": { 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "port", 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8086", 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.503 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server port", 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "false", 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "threads": { 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threads", 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.505 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "32", 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "username": { 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "username", 2024-03-19T02:03:45.506 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "username of InfluxDB server user", 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "verify_ssl": { 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "verify_ssl", 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.507 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "true", 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "insights", 2024-03-19T02:03:45.508 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.509 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.510 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.511 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "iostat", 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.514 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.515 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.516 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.517 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.518 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "k8sevents", 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_event_retention_days": { 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ceph_event_retention_days", 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "config_check_secs": { 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_check_secs", 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.520 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "10", 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.521 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.522 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.524 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.525 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "localpool", 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "failure_domain": { 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "failure_domain", 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.526 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "host", 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "failure domain for any created local pool", 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.527 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.528 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.529 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.530 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.531 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": { 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_size", 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.532 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default min_size for any created local pool", 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "num_rep": { 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "num_rep", 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.533 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default replica count for any created local pool", 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.534 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": { 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_num", 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "128", 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default pg_num for any created local pool", 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.535 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "prefix": { 2024-03-19T02:03:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prefix", 2024-03-19T02:03:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.536 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name prefix for any created local pool", 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "subtree": { 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "subtree", 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "rack", 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.537 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mds_autoscaler", 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.539 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.541 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.542 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirroring", 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.543 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.544 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.545 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.546 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "nfs", 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.547 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.548 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.549 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.550 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.551 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "fail_fs": { 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "fail_fs", 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.553 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.554 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.555 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.556 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.557 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.558 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator": { 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "rook", 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: "test_orchestrator" 2024-03-19T02:03:45.559 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Orchestrator backend", 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_perf_query", 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.561 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.562 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.563 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_support", 2024-03-19T02:03:45.565 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.566 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.567 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.569 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.570 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_autoscaler", 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.571 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.572 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.573 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.574 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.575 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.576 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.577 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "threshold": { 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threshold", 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1.0", 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "scaling threshold", 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "progress", 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_pg_recovery_event": { 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_pg_recovery_event", 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.580 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.581 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.582 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.583 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.586 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.587 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "max_completed_events": { 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_completed_events", 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.588 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "50", 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "number of past completed events to remember", 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long the module is going to sleep", 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus", 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.590 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "cache": { 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cache", 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "exclude_perf_counters": { 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "exclude_perf_counters", 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.592 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.593 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.595 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.596 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.597 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools": { 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools", 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.598 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "300", 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_interval": { 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_interval", 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15.0", 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.600 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.601 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "9283", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.602 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "stale_cache_strategy": { 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stale_cache_strategy", 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "log", 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.603 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "default", 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "default", 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "error" 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.604 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.605 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.606 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.607 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.608 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.609 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.610 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_snap_create": { 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_snap_create", 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.611 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "mirror_snapshot_schedule": { 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirror_snapshot_schedule", 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.612 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "trash_purge_schedule": { 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "trash_purge_schedule", 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "restful", 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_auth": { 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_auth", 2024-03-19T02:03:45.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.615 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.616 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.617 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.619 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.620 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.621 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.622 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rgw", 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rook", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "storage_class": { 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "storage_class", 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "local", 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "selftest", 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "roption1": { 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption1", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "roption2": { 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption2", 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "xyz", 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption1": { 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption1", 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption2": { 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption2", 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption3": { 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption3", 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption4": { 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption4", 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption5": { 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption5", 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption6": { 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption6", 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption7": { 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption7", 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "42", 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "testkey": { 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testkey", 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "testlkey": { 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testlkey", 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "testnewline": { 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testnewline", 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snap_schedule", 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_m_granularity": { 2024-03-19T02:03:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_m_granularity", 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow minute scheduled snapshots", 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "dump_on_update": { 2024-03-19T02:03:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dump_on_update", 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "dump database to debug log on update", 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stats", 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "status", 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telegraf", 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "address": { 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "address", 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15", 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telemetry", 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_basic": { 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_basic", 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_crash": { 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_crash", 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_device": { 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_device", 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_ident": { 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_ident", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_perf": { 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_perf", 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "contact": { 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "contact", 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": { 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "description", 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "device_url": { 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_url", 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "24", 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "8", 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "last_opt_revision": { 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "last_opt_revision", 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1", 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard": { 2024-03-19T02:03:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard", 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard_description": { 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard_description", 2024-03-19T02:03:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "organization": { 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "organization", 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "proxy": { 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "proxy", 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "url": { 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url", 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "test_orchestrator", 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_clones": { 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_clones", 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4", 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "periodic_async_work": { 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "periodic_async_work", 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Periodically check for async work", 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_delay": { 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_delay", 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_no_wait": { 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_no_wait", 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix", 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "discovery_interval": { 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "discovery_interval", 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "100", 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "identifier": { 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "identifier", 2024-03-19T02:03:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_host": { 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_host", 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_port": { 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_port", 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10051", 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_sender": { 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_sender", 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "modules": [ 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "dashboard", 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "iostat", 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "nfs", 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "restful" 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "available_modules": [ 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "alerts", 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How frequently to reexamine health status", 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_destination": { 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_destination", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email address to send alerts to", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_from_name": { 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_from_name", 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "Ceph", 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Email From: name", 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_host": { 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_host", 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP server", 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_password": { 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_password", 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Password to authenticate with", 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_port": { 2024-03-19T02:03:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_port", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "465", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP port", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_sender": { 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_sender", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SMTP envelope sender", 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_ssl": { 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_ssl", 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "smtp_user": { 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "smtp_user", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "User to authenticate as", 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "balancer", 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "active": { 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "active", 2024-03-19T02:03:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically balance PGs across cluster", 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_time": { 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_time", 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0000", 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "beginning time of day to automatically balance", 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "begin_weekday": { 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "begin_weekday", 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_max_iterations": { 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_max_iterations", 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "25", 2024-03-19T02:03:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "250", 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_metrics": { 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_metrics", 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "pgs,objects,bytes", 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_compat_step": { 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crush_compat_step", 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.5", 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0.001", 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "0.999", 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "aggressiveness of optimization", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "end_time": { 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_time", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2359", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ending time of day to automatically balance", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-19T02:03:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "end_weekday": { 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "end_weekday", 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "0", 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "6", 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "min_score": { 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_score", 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "upmap", 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-compat", 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "none", 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "read", 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap", 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap-read" 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Balancer mode", 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_ids": { 2024-03-19T02:03:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_ids", 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_deviation": { 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_deviation", 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "upmap_max_optimizations": { 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "upmap_max_optimizations", 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm", 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_down_multiplier": { 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_down_multiplier", 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_refresh_rate": { 2024-03-19T02:03:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_refresh_rate", 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "20", 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "agent_starting_port": { 2024-03-19T02:03:45.754 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "agent_starting_port", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4721", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_ptrace": { 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_ptrace", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.755 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_interval": { 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_interval", 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.756 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "autotune_memory_target_ratio": { 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "autotune_memory_target_ratio", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0.7", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.757 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_log_destination": { 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephadm_log_destination", 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.758 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "file", 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "file,syslog", 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "syslog" 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "cgroups_split": { 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cgroups_split", 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.759 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "config_checks_enabled": { 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_checks_enabled", 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.760 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "config_dashboard": { 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_dashboard", 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_alertmanager": { 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_alertmanager", 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-19T02:03:45.762 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_base": { 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_base", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.763 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Container image name, without the tag", 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_elasticsearch": { 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_elasticsearch", 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.764 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "elasticsearch container image", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_grafana": { 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_grafana", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.765 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_haproxy": { 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_haproxy", 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "HAproxy container image", 2024-03-19T02:03:45.766 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_agent": { 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_agent", 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger agent container image", 2024-03-19T02:03:45.767 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_collector": { 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_collector", 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger collector container image", 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_jaeger_query": { 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_jaeger_query", 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Jaeger query container image", 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.769 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_keepalived": { 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_keepalived", 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Keepalived container image", 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.770 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_loki": { 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_loki", 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Loki container image", 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.771 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_node_exporter": { 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_node_exporter", 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_nvmeof": { 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_nvmeof", 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Nvme-of container image", 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_prometheus": { 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_prometheus", 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Prometheus container image", 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.774 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_promtail": { 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_promtail", 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Promtail container image", 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image_snmp_gateway": { 2024-03-19T02:03:45.775 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_image_snmp_gateway", 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "SNMP Gateway container image", 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "container_init": { 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "container_init", 2024-03-19T02:03:45.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Run podman/docker with `--init`", 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "daemon_cache_timeout": { 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "daemon_cache_timeout", 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.777 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "default_cephadm_command_timeout": { 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_cephadm_command_timeout", 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.778 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "900", 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "default_registry": { 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "default_registry", 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "docker.io", 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "device_cache_timeout": { 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_cache_timeout", 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.780 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1800", 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache device inventory", 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "device_enhanced_scan": { 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_enhanced_scan", 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.781 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "facts_cache_timeout": { 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "facts_cache_timeout", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.782 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "seconds to cache host facts data", 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "host_check_interval": { 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "host_check_interval", 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.783 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to perform a host check", 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "hw_monitoring": { 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hw_monitoring", 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-19T02:03:45.784 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "inventory_list_all": { 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "inventory_list_all", 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.786 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "log_refresh_metadata": { 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_refresh_metadata", 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.787 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.789 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-19T02:03:45.791 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-19T02:03:45.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "*", 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "max_count_per_host": { 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_count_per_host", 2024-03-19T02:03:45.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of daemons per service per host", 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd_draining_count": { 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_osd_draining_count", 2024-03-19T02:03:45.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "migration_current": { 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "migration_current", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "internal - do not modify", 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": { 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mode", 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.796 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "root", 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm-package", 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "root" 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "mode for remote execution of cephadm", 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "oob_default_addr": { 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "oob_default_addr", 2024-03-19T02:03:45.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "169.254.1.1", 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "prometheus_alerts_path": { 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus_alerts_path", 2024-03-19T02:03:45.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_insecure": { 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_insecure", 2024-03-19T02:03:45.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_password": { 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_password", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_url": { 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_url", 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.801 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "registry_username": { 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "registry_username", 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.802 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "secure_monitoring_stack": { 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "secure_monitoring_stack", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.803 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "service_discovery_port": { 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "service_discovery_port", 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8765", 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.804 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "cephadm service discovery port", 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "ssh_config_file": { 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssh_config_file", 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-19T02:03:45.805 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "use_agent": { 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_agent", 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "use_repo_digest": { 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "use_repo_digest", 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_failed_host_check": { 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_failed_host_check", 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if the host check fails", 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.808 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_daemons": { 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_daemons", 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_on_stray_hosts": { 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_on_stray_hosts", 2024-03-19T02:03:45.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crash", 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.811 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.812 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.813 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "retain_interval": { 2024-03-19T02:03:45.815 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retain_interval", 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "31536000", 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_recent_interval": { 2024-03-19T02:03:45.816 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_recent_interval", 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1209600", 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dashboard", 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.818 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_HOST": { 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.819 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.820 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_ENABLED": { 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_ENABLED", 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "ENABLE_BROWSABLE_API": { 2024-03-19T02:03:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_PASSWORD": { 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-03-19T02:03:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-19T02:03:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_URL": { 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_URL", 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_API_USERNAME": { 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-19T02:03:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-19T02:03:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "MULTICLUSTER_CONFIG": { 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "{}", 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_HOST": { 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-19T02:03:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-19T02:03:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-19T02:03:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-19T02:03:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-19T02:03:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-19T02:03:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-19T02:03:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-19T02:03:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_ENABLED": { 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8", 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "45", 2024-03-19T02:03:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ACCESS_KEY": { 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "admin", 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SECRET_KEY": { 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "RGW_API_SSL_VERIFY": { 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "UNSAFE_TLS_v1_2": { 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "cross_origin_url": { 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cross_origin_url", 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "crt_file": { 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "crt_file", 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "debug": { 2024-03-19T02:03:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "debug", 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Enable/disable debug options", 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "jwt_token_ttl": { 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "jwt_token_ttl", 2024-03-19T02:03:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "28800", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "motd": { 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "motd", 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "The message of the day", 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect_resolve_ip_addr": { 2024-03-19T02:03:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "redirect_resolve_ip_addr", 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8080", 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl_server_port": { 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl_server_port", 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8443", 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "redirect", 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "redirect" 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "url_prefix": { 2024-03-19T02:03:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url_prefix", 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_monitoring": { 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_monitoring", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "monitor device health metrics", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "mark_out_threshold": { 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mark_out_threshold", 2024-03-19T02:03:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "2419200", 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": { 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pool_name", 2024-03-19T02:03:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "device_health_metrics", 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "retention_period": { 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "retention_period", 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15552000", 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long to retain device health metrics", 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_frequency": { 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_frequency", 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "self_heal": { 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "self_heal", 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how frequently to wake up and check device health", 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "warn_threshold": { 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "warn_threshold", 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7257600", 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "diskprediction_local", 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "predict_interval": { 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predict_interval", 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "86400", 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "predictor_model": { 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "predictor_model", 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "prophetstor", 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "600", 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "influx", 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": false, 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "influxdb python module not found", 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "batch_size": { 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "batch_size", 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5000", 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "database": { 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "database", 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "ceph", 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-19T02:03:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": { 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "hostname", 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server hostname", 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "30", 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "5", 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "password": { 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "password", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "password of InfluxDB server user", 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "port": { 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "port", 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "8086", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "InfluxDB server port", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "ssl": { 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ssl", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "false", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "threads": { 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threads", 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "32", 2024-03-19T02:03:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "username": { 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "username", 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "username of InfluxDB server user", 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "verify_ssl": { 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "verify_ssl", 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "true", 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-19T02:03:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "insights", 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "iostat", 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "k8sevents", 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_event_retention_days": { 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "ceph_event_retention_days", 2024-03-19T02:03:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "7", 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "config_check_secs": { 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "config_check_secs", 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "10", 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "localpool", 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "failure_domain": { 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "failure_domain", 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "host", 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "failure domain for any created local pool", 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.935 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": { 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "min_size", 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default min_size for any created local pool", 2024-03-19T02:03:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "num_rep": { 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "num_rep", 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3", 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default replica count for any created local pool", 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": { 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_num", 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "128", 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "default pg_num for any created local pool", 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "prefix": { 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prefix", 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "name prefix for any created local pool", 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "subtree": { 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "subtree", 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "rack", 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mds_autoscaler", 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirroring", 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "nfs", 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.954 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "fail_fs": { 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "fail_fs", 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator": { 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "orchestrator", 2024-03-19T02:03:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm", 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "rook", 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "test_orchestrator" 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Orchestrator backend", 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_perf_query", 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.971 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "osd_support", 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.972 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.973 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.975 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "pg_autoscaler", 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.977 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.978 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.980 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.981 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "threshold": { 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "threshold", 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "3.0", 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1.0", 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "scaling threshold", 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "progress", 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_pg_recovery_event": { 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_pg_recovery_event", 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.984 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.985 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.986 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.988 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.989 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "max_completed_events": { 2024-03-19T02:03:45.990 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_completed_events", 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "50", 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "number of past completed events to remember", 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "sleep_interval": { 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "sleep_interval", 2024-03-19T02:03:45.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "5", 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "how long the module is going to sleep", 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:45.992 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "prometheus", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "cache": { 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cache", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "exclude_perf_counters": { 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "exclude_perf_counters", 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.996 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:45.997 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.998 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools": { 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools", 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:45.999 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.000 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "300", 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "scrape_interval": { 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "scrape_interval", 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15.0", 2024-03-19T02:03:46.001 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.002 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "::", 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "9283", 2024-03-19T02:03:46.003 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "stale_cache_strategy": { 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stale_cache_strategy", 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "log", 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_behaviour": { 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_behaviour", 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "default", 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.005 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "default", 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "error" 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "standby_error_status_code": { 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "standby_error_status_code", 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.006 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "500", 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "400", 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "599", 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.007 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.008 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.010 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.011 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_snap_create": { 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_snap_create", 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10", 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mirror_snapshot_schedule": { 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mirror_snapshot_schedule", 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.013 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "trash_purge_schedule": { 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "trash_purge_schedule", 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.014 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "restful", 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.015 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_auth": { 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enable_auth", 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "key_file": { 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "key_file", 2024-03-19T02:03:46.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.018 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.019 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.020 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.021 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "server_addr": { 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_addr", 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.022 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "server_port": { 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "server_port", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.023 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rgw", 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.024 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.025 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.027 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.028 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rook", 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.029 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.030 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.033 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.034 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "storage_class": { 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "storage_class", 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "local", 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.035 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "selftest", 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.036 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.037 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.038 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.040 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "roption1": { 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption1", 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.041 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "roption2": { 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "roption2", 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "xyz", 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.042 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption1": { 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption1", 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption2": { 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption2", 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption3": { 2024-03-19T02:03:46.044 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption3", 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "float", 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption4": { 2024-03-19T02:03:46.045 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption4", 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.046 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption5": { 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption5", 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption6": { 2024-03-19T02:03:46.047 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption6", 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "rwoption7": { 2024-03-19T02:03:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rwoption7", 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "1", 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "42", 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "testkey": { 2024-03-19T02:03:46.049 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testkey", 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "testlkey": { 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testlkey", 2024-03-19T02:03:46.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "testnewline": { 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "testnewline", 2024-03-19T02:03:46.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snap_schedule", 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_m_granularity": { 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "allow_m_granularity", 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "allow minute scheduled snapshots", 2024-03-19T02:03:46.053 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "dump_on_update": { 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "dump_on_update", 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "dump database to debug log on update", 2024-03-19T02:03:46.054 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.055 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.056 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.057 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "stats", 2024-03-19T02:03:46.059 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.060 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.061 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.062 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.064 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "status", 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.065 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.066 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.067 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.069 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telegraf", 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "address": { 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "address", 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.071 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "15", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.072 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.073 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.074 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.075 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.076 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "telemetry", 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.077 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_basic": { 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_basic", 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.078 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_crash": { 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_crash", 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.079 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_device": { 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_device", 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.080 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_ident": { 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_ident", 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.081 INFO:teuthology.orchestra.run.smithi092.stdout: "channel_perf": { 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "channel_perf", 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "contact": { 2024-03-19T02:03:46.082 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "contact", 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": { 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "description", 2024-03-19T02:03:46.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "device_url": { 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "device_url", 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.084 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "enabled": { 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "enabled", 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.085 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.086 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "24", 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "8", 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "last_opt_revision": { 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "last_opt_revision", 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "1", 2024-03-19T02:03:46.087 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard": { 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard", 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.088 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "leaderboard_description": { 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "leaderboard_description", 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.089 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.090 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.092 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.093 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "organization": { 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "organization", 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.095 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "proxy": { 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "proxy", 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "url": { 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "url", 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "test_orchestrator", 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.098 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.099 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.100 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.101 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.103 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.104 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.105 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.106 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "max_concurrent_clones": { 2024-03-19T02:03:46.107 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "max_concurrent_clones", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "4", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "periodic_async_work": { 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "periodic_async_work", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.108 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Periodically check for async work", 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_delay": { 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_delay", 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "0", 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "snapshot_clone_no_wait": { 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "snapshot_clone_no_wait", 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "True", 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix", 2024-03-19T02:03:46.111 INFO:teuthology.orchestra.run.smithi092.stdout: "can_run": true, 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "error_string": "", 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "module_options": { 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "discovery_interval": { 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "discovery_interval", 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "uint", 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "100", 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.112 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "identifier": { 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "identifier", 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.113 INFO:teuthology.orchestra.run.smithi092.stdout: "interval": { 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "interval", 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "secs", 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "60", 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.114 INFO:teuthology.orchestra.run.smithi092.stdout: "log_level": { 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_level", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.115 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster": { 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster", 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_cluster_level": { 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_cluster_level", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "info", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [ 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "critical", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "debug", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "error", 2024-03-19T02:03:46.117 INFO:teuthology.orchestra.run.smithi092.stdout: "info", 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "warning" 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": { 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "log_to_file", 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "bool", 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "False", 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.118 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_host": { 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_host", 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "", 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_port": { 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_port", 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "int", 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "10051", 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "zabbix_sender": { 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "zabbix_sender", 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "str", 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "level": "advanced", 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 0, 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "min": "", 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "max": "", 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "enum_allowed": [], 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "desc": "", 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "long_desc": "", 2024-03-19T02:03:46.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tags": [], 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "see_also": [] 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "services": { 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "dashboard": "https://172.21.15.92:8443/" 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "always_on_modules": { 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "octopus": [ 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-03-19T02:03:46.122 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "pacific": [ 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-03-19T02:03:46.123 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "quincy": [ 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-03-19T02:03:46.124 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "reef": [ 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "squid": [ 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "balancer", 2024-03-19T02:03:46.125 INFO:teuthology.orchestra.run.smithi092.stdout: "crash", 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "devicehealth", 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "orchestrator", 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscaler", 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "progress", 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_support", 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "status", 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "telemetry", 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "volumes" 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "last_failure_osd_epoch": 3, 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "active_clients": [ 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "devicehealth", 2024-03-19T02:03:46.126 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2292169595 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "libcephsqlite", 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-03-19T02:03:46.127 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2477781456 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "rbd_support", 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 206244231 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:46.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volumes", 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4250800294 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-03-19T02:03:46.129 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T02:03:46.131 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:44 smithi144 ceph-mon[67199]: pgmap v679: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-03-19T02:03:46.131 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:44 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/364822246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T02:03:46.131 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:44 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1493311694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T02:03:46.131 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:44 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2655405219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T02:03:46.131 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:45 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/63654038' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-19T02:03:46.131 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:44 smithi159 ceph-mon[67110]: pgmap v679: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-03-19T02:03:46.131 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:44 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/364822246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T02:03:46.132 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:44 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1493311694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T02:03:46.132 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:44 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2655405219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T02:03:46.132 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:45 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/63654038' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-19T02:03:46.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6fd00a2800 msgr2=0x7f6fd00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6fd00a2800 0x7f6fd00a4cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc000c00 tx=0x7f6fdc005fd0 comp rx=0 tx=0).stop 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8103410 msgr2=0x7f6fe819f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8103410 0x7f6fe819f180 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000efd0 tx=0x7f6fe000c6a0 comp rx=0 tx=0).stop 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 shutdown_connections 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6fe8103410 0x7f6fe819f180 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f6fd00a2800 0x7f6fd00a4cc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fe8102a50 0x7f6fe8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 --2- 172.21.15.92:0/63654038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fe8101860 0x7f6fe8197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 >> 172.21.15.92:0/63654038 conn(0x7f6fe80fcdd0 msgr2=0x7f6fe8105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 shutdown_connections 2024-03-19T02:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:03:44.776+0000 7f6fee8ac640 1 -- 172.21.15.92:0/63654038 wait complete. 2024-03-19T02:03:46.139 DEBUG:tasks.check_counter:Getting stats from a 2024-03-19T02:03:46.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-19T02:03:46.139 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:44 smithi092 ceph-mon[62574]: pgmap v679: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-03-19T02:03:46.140 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:44 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/364822246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T02:03:46.140 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:44 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1493311694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T02:03:46.140 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:44 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2655405219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T02:03:46.140 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:45 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/63654038' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-19T02:03:46.386 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T02:03:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:46 smithi092 ceph-mon[62574]: pgmap v680: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 16 op/s 2024-03-19T02:03:46.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:46 smithi092 ceph-mon[62574]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T02:03:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:46 smithi159 ceph-mon[67110]: pgmap v680: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 16 op/s 2024-03-19T02:03:46.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:46 smithi159 ceph-mon[67110]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T02:03:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:46 smithi144 ceph-mon[67199]: pgmap v680: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 16 op/s 2024-03-19T02:03:47.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:46 smithi144 ceph-mon[67199]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-19T02:03:48.322 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T02:03:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:03:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 189, 2024-03-19T02:03:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 428, 2024-03-19T02:03:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 42123, 2024-03-19T02:03:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 71994, 2024-03-19T02:03:48.322 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 8, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.028165254, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.010429461, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.010963331, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.003118930, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 426, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.029220203, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000068592 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 580, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000118232, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000203 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.323 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 278, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 265, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 817289, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 193821, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 10, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.030305357, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.009610274, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.021800655, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.003592156, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 265, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.015450383, 2024-03-19T02:03:48.324 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000058303 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000676, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000169 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 814208, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 137056 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 16, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 176, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 4241, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 373205, 2024-03-19T02:03:48.325 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.013910333, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.007400364, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.005209067, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000311971, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 174, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.033433271, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000192145 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.326 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 365840 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 47, 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001019880, 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000021699 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:48.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000009085, 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000004542 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 11, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 10, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 10, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 10, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-03-19T02:03:48.329 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 1, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-03-19T02:03:48.330 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 4, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 234, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-03-19T02:03:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 1, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-03-19T02:03:48.332 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-03-19T02:03:48.333 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 2, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 2, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 2, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-03-19T02:03:48.334 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 2, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 4194892, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001116189, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001116189 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-03-19T02:03:48.335 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 13, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 13, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 0, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 11, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 12, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 1, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 10, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 10, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 0, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 51768, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 223612 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-03-19T02:03:48.336 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.337 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-03-19T02:03:48.338 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.339 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-03-19T02:03:48.340 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.341 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.342 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.345 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.346 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 0, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 0, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 0, 2024-03-19T02:03:48.347 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:03:48.348 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:03:48.349 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 238039, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 75, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:03:48.350 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7528, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 158, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 50664, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 198, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-03-19T02:03:48.351 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 35, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 10710, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 35, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 35, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.200994226, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005742692 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.352 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 35, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 57 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 35, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 0, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 35, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 0, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 1, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 5, 2024-03-19T02:03:48.353 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 6, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-03-19T02:03:48.354 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-03-19T02:03:48.355 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 87, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 2, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 9, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 9, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 14, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 0, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 1 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-03-19T02:03:48.356 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:03:48.357 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 479, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 804330, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 479, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 479, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 804330, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 35, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 10710, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 35, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 10710, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 35, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 35, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 35, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 35, 2024-03-19T02:03:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-19T02:03:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55f75c35a500": { 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 2, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 588, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 2, 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1, 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 588, 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:48.363 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T02:03:48.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:48 smithi159 ceph-mon[67110]: pgmap v681: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 16 op/s 2024-03-19T02:03:49.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:48 smithi092 ceph-mon[62574]: pgmap v681: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 16 op/s 2024-03-19T02:03:49.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:48 smithi144 ceph-mon[67199]: pgmap v681: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 16 op/s 2024-03-19T02:03:49.149 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-19T02:03:49.149 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-19T02:03:49.149 DEBUG:tasks.check_counter:Getting stats from d 2024-03-19T02:03:49.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-19T02:03:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T02:03:50.920 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:50 smithi092 ceph-mon[62574]: pgmap v682: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T02:03:50.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:50 smithi159 ceph-mon[67110]: pgmap v682: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T02:03:51.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:50 smithi144 ceph-mon[67199]: pgmap v682: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 19 op/s 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 0, 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2, 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 0, 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 0, 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.001132389, 2024-03-19T02:03:51.327 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000000000, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.000273902, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.328 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 223, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 218, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 752080, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 133824, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.027220803, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.008936928, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.018462041, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.003556775, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 217, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.006811120, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000031387 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 752080, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 133824 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 5, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 167, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 3184, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 107232, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 3, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.009295638, 2024-03-19T02:03:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.006756327, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001583832, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 166, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.007129832, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000042950 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 107232 2024-03-19T02:03:51.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:03:51.332 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-03-19T02:03:51.333 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 149007, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 30, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-03-19T02:03:51.334 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 928, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-03-19T02:03:51.335 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 224, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 720597, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 224, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 224, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 720597, 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:51.336 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:51.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:51.337 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T02:03:52.064 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-19T02:03:52.064 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-19T02:03:52.064 DEBUG:tasks.check_counter:Getting stats from g 2024-03-19T02:03:52.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-19T02:03:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T02:03:52.783 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:52 smithi092 ceph-mon[62574]: pgmap v683: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.3 KiB/s wr, 15 op/s 2024-03-19T02:03:52.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:52 smithi159 ceph-mon[67110]: pgmap v683: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.3 KiB/s wr, 15 op/s 2024-03-19T02:03:53.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:52 smithi144 ceph-mon[67199]: pgmap v683: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.3 KiB/s wr, 15 op/s 2024-03-19T02:03:54.105 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T02:03:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:03:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1007, 2024-03-19T02:03:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 998, 2024-03-19T02:03:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 1055440, 2024-03-19T02:03:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 379534, 2024-03-19T02:03:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-03-19T02:03:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.085557318, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.030953788, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.030662218, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.021654314, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 997, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.032925762, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000033024 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 794368, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 132448 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 780, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 942, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 204878, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 615587, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.053102022, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.020766182, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.012808832, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.014630998, 2024-03-19T02:03:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 940, 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.021272443, 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000022630 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 367808 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 8, 2024-03-19T02:03:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 8, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 4653, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 1732, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.001642305, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000103098, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001395495, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000074615, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002572776, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000367539 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3113, 2024-03-19T02:03:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.060829919, 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000019540 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-03-19T02:03:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 0, 2024-03-19T02:03:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 1, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-03-19T02:03:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:03:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:03:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-03-19T02:03:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 2, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 2, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 4194892, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 4194352, 2024-03-19T02:03:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 2 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 3, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 3, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 0, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 1, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 1, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 0, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 0, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 0, 2024-03-19T02:03:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 49948, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 215420 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-03-19T02:03:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-03-19T02:03:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 0, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 0, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 0, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 0, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 0, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-03-19T02:03:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:03:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:03:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 197543, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 56, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:03:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7528, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 158, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 8464, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 13, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 1560, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 21, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 3, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 1560, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1560, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.623982232, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000399988 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-03-19T02:03:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1560, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1561 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 1560, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 1560, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 777, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 781, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-03-19T02:03:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 1, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-03-19T02:03:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-03-19T02:03:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 87, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 2, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 6, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 6, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-03-19T02:03:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-03-19T02:03:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 1791, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1088980, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 1791, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1791, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1088980, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1560, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 593, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1560, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 593, 2024-03-19T02:03:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1560, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 1560, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1560, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1560, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-19T02:03:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x561e0421c6e0": { 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-03-19T02:03:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:54.147 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T02:03:54.875 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-19T02:03:54.875 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-19T02:03:54.875 DEBUG:tasks.check_counter:Getting stats from j 2024-03-19T02:03:54.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-19T02:03:54.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:54 smithi159 ceph-mon[67110]: pgmap v684: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 KiB/s wr, 18 op/s 2024-03-19T02:03:55.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:54 smithi092 ceph-mon[62574]: pgmap v684: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 KiB/s wr, 18 op/s 2024-03-19T02:03:55.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:54 smithi144 ceph-mon[67199]: pgmap v684: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 KiB/s wr, 18 op/s 2024-03-19T02:03:55.124 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.a/config 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 0, 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2, 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 0, 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 0, 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 0, 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.001197919, 2024-03-19T02:03:56.886 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.000000000, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.000282228, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T02:03:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 229, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 222, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 755648, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 135552, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 2, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 1, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.026318163, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.008802243, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.017406107, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.003585641, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 222, 2024-03-19T02:03:56.888 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.007655520, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000034484 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 755648, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 135552 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 167, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 160, 2024-03-19T02:03:56.889 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 107136, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 3, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.009513294, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.006704591, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001699434, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 165, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.006669147, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000040419 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:56.890 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 107136 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 0, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:03:56.891 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-03-19T02:03:56.892 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:03:56.893 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 149007, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 30, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 0, 2024-03-19T02:03:56.894 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 928, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 2, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-03-19T02:03:56.895 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 226, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 720857, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 226, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 226, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 720857, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-03-19T02:03:56.896 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-03-19T02:03:56.897 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:56 smithi092 ceph-mon[62574]: pgmap v685: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-19T02:03:57.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:56 smithi144 ceph-mon[67199]: pgmap v685: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-19T02:03:57.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:56 smithi159 ceph-mon[67110]: pgmap v685: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-19T02:03:57.635 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-19T02:03:57.635 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-19T02:03:57.636 DEBUG:tasks.check_counter:Getting stats from b 2024-03-19T02:03:57.636 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-19T02:03:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T02:03:59.044 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:03:58 smithi144 ceph-mon[67199]: pgmap v686: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-19T02:03:59.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:03:58 smithi092 ceph-mon[62574]: pgmap v686: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-19T02:03:59.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:03:58 smithi159 ceph-mon[67110]: pgmap v686: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-03-19T02:03:59.786 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-19T02:03:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:03:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 222, 2024-03-19T02:03:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 476, 2024-03-19T02:03:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 55697, 2024-03-19T02:03:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 111122, 2024-03-19T02:03:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 8, 2024-03-19T02:03:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.028307578, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.009893367, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.010868906, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.003649276, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 474, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.025268917, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000053309 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 630, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000132973, 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000211 2024-03-19T02:03:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 32, 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 192, 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 9992, 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 427692, 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 12, 2024-03-19T02:03:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 7, 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.010634393, 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.004835159, 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.004575133, 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000180622, 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 191, 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.026528893, 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000138894 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 28, 2024-03-19T02:03:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000002411, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000086 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 6624, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 375696 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 279, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 262, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 832443, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 147892, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.028963356, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.008186316, 2024-03-19T02:03:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.022998467, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.003822719, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 261, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.015899137, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000060916 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 829632, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 137888 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 35, 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.001198322, 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000034237 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-19T02:03:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000007699, 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000003849 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 5, 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 3, 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3, 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.002350114, 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000783371 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-19T02:03:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 2, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 11, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 12, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 12, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 11, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-19T02:03:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 2, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 8, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 6, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 2, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-19T02:03:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 4, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 234, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T02:03:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 3, 2024-03-19T02:03:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-19T02:03:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 6, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-19T02:03:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 6, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 2, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4195956, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5, 2024-03-19T02:03:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.007579123, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001515824 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 15, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 15, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 0, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 13, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 14, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 1, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 12, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 12, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-03-19T02:03:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 3, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 3, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 49712, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 223612 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 10, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 5, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 8, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 1, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.001483400, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001483400 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2, 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000866714, 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000433357 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-19T02:03:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-19T02:03:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-19T02:03:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-19T02:03:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-19T02:03:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 0, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 2, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 0, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 2, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 2, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 74, 2024-03-19T02:03:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:03:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:03:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:03:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 239271, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 84, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 176, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 2, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7528, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 158, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:03:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 59120, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 237, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 44, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 12269, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 33, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 44, 2024-03-19T02:03:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 44, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.205996391, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.004681736 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 44, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 68 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 44, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 0, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 44, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-19T02:03:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 10, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 5, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 8, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 6, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 0, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 20, 2024-03-19T02:03:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-19T02:03:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 87, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 2, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 9, 2024-03-19T02:03:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 9, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 17, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 2 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 529, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 833352, 2024-03-19T02:03:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 529, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 529, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 833352, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-19T02:03:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 44, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 12269, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 44, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 12269, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 44, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 44, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 44, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 44, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x555b0456c500": { 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 6, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1652, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 6, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 5, 2024-03-19T02:03:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1652, 2024-03-19T02:03:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:03:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:03:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:03:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:03:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:03:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:03:59.831 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-19T02:04:00.491 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-19T02:04:00.492 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-19T02:04:00.492 DEBUG:tasks.check_counter:Getting stats from e 2024-03-19T02:04:00.492 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-19T02:04:00.769 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:00 smithi144 ceph-mon[67199]: pgmap v687: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-03-19T02:04:00.769 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T02:04:01.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:00 smithi092 ceph-mon[62574]: pgmap v687: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-03-19T02:04:01.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:00 smithi159 ceph-mon[67110]: pgmap v687: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2024-03-19T02:04:02.505 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1396, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1398, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 2647437, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 443992, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 6, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 2, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.078798177, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.027810458, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.020786109, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.023039818, 2024-03-19T02:04:02.506 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1396, 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.026740987, 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000019155 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:04:02.507 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 907, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 895, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 978174, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 348717, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.065363278, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.019223545, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.026260734, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.019726739, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 894, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.021877516, 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000024471 2024-03-19T02:04:02.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 813824, 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 136288 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 791, 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 955, 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 267280, 2024-03-19T02:04:02.509 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 620524, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.060499079, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.020734641, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.014824772, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.018590336, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 954, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.033261218, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000034865 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 370576 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-19T02:04:02.511 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 4965, 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.082076181, 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000016530 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-19T02:04:02.512 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 2, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-19T02:04:02.513 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 2, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 2, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 1, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-19T02:04:02.514 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-19T02:04:02.515 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-19T02:04:02.516 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-19T02:04:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:04:02.518 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 1969, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-19T02:04:02.519 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 3, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 6485062, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 6484463, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 1969 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-19T02:04:02.520 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 4, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 13, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 9, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 2, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 11, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 9, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 2, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 11, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 9, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 48912, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215420 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.521 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.522 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-19T02:04:02.523 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.524 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.525 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.526 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.526 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-19T02:04:02.526 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.526 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.527 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-19T02:04:02.528 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-19T02:04:02.529 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.530 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.532 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 1, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 0, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-03-19T02:04:02.533 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 4293, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:04:02.534 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:04:02.535 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 189383, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 50, 2024-03-19T02:04:02.536 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7528, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 158, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 13392, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 36, 2024-03-19T02:04:02.537 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 2850, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 2850, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2850, 2024-03-19T02:04:02.538 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.067982157, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000374730 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2850, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2851 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 2850, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 2850, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 1452, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-03-19T02:04:02.539 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 1396, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-19T02:04:02.540 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-19T02:04:02.541 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 87, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 2, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 5, 2024-03-19T02:04:02.542 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 5, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-19T02:04:02.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 3090, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 3599551, 2024-03-19T02:04:02.544 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 3090, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 3090, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 3599551, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:02.545 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2850, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 2290763, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2850, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2290763, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:02.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2850, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 2850, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2850, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2850, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.547 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-19T02:04:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5559bd7906e0": { 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:02.549 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:02.550 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-19T02:04:02.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:02 smithi144 ceph-mon[67199]: pgmap v688: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-19T02:04:03.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:02 smithi092 ceph-mon[62574]: pgmap v688: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-19T02:04:03.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:02 smithi159 ceph-mon[67110]: pgmap v688: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-19T02:04:03.309 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-19T02:04:03.310 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-19T02:04:03.310 DEBUG:tasks.check_counter:Getting stats from h 2024-03-19T02:04:03.310 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-19T02:04:03.557 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T02:04:05.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:04 smithi092 ceph-mon[62574]: pgmap v689: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-19T02:04:05.082 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:04 smithi144 ceph-mon[67199]: pgmap v689: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-19T02:04:05.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:04 smithi159 ceph-mon[67110]: pgmap v689: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-19T02:04:05.320 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1703, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1705, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1296253, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 541564, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 6, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 2, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.103347883, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.037132780, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.025753334, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.031503650, 2024-03-19T02:04:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1703, 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.032453819, 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000019056 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:04:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 247, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 233, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 818392, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 137594, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.027665616, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.008216343, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.019813506, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.003705053, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 233, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.011595288, 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000049765 2024-03-19T02:04:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 816736, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 136256 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 781, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 946, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 192111, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 619609, 2024-03-19T02:04:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.054975099, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.018446630, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.011862176, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.019446936, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 944, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.021766531, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000023057 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 371568 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 4167, 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.079970506, 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000019191 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 16, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 4, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 12, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-19T02:04:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 9, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 2, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-19T02:04:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 4, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 234, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:04:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-19T02:04:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:04:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-19T02:04:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 168, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 51, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4952984, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4987364, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 4986765, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 168 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 14, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 17, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 3, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 9, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 11, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 2, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 16, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 18, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 2, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-19T02:04:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 47688, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215420 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-19T02:04:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-19T02:04:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-19T02:04:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-19T02:04:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-19T02:04:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-19T02:04:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 3, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 2, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 2, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 7926, 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-19T02:04:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:04:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:04:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 189463, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 52, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-19T02:04:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7528, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 158, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 53890, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 253, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-19T02:04:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 2487, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 26, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 4, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 2487, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2487, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.839985863, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000337750 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2487, 2024-03-19T02:04:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2488 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 2487, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 2487, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 1568, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 916, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-19T02:04:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 2, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:04:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-19T02:04:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 87, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 2, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 5, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 5, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-03-19T02:04:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 2727, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 2046092, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 2727, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2727, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2046092, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2487, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 793070, 2024-03-19T02:04:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2487, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 793070, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 2487, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 2487, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2487, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 2487, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-19T02:04:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x55f83ad9e6e0": { 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:05.364 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-19T02:04:06.044 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-19T02:04:06.045 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-19T02:04:06.045 DEBUG:tasks.check_counter:Getting stats from k 2024-03-19T02:04:06.045 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-19T02:04:06.292 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.b/config 2024-03-19T02:04:07.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:06 smithi092 ceph-mon[62574]: pgmap v690: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-19T02:04:07.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:06 smithi144 ceph-mon[67199]: pgmap v690: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-19T02:04:07.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:06 smithi159 ceph-mon[67110]: pgmap v690: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-19T02:04:08.016 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-19T02:04:08.016 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:04:08.016 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 1039, 2024-03-19T02:04:08.016 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1025, 2024-03-19T02:04:08.016 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1084048, 2024-03-19T02:04:08.016 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 391152, 2024-03-19T02:04:08.016 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 1, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.087449824, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.029258547, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.034343612, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.022354848, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1025, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.032500383, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000031707 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.017 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 817936, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 139296 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 4, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 173, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 1550, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 373294, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.007830713, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.005076653, 2024-03-19T02:04:08.018 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.001428877, 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000083403, 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 171, 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.009061933, 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000052993 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 372272 2024-03-19T02:04:08.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 795, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 797, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 208887, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 252811, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 4, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.045875496, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.013899771, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.012863957, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.014732950, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 795, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.016245887, 2024-03-19T02:04:08.020 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000020435 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-19T02:04:08.021 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3173, 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.064796798, 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000020421 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:08.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-19T02:04:08.023 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 1, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-19T02:04:08.024 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-03-19T02:04:08.025 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:04:08.026 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:04:08.027 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-19T02:04:08.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 2, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 2, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-19T02:04:08.029 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 4194892, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 4194352, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 2 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 3, 2024-03-19T02:04:08.030 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 3, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 0, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 1, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 1, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 0, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 0, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 0, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 0, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 49916, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215420 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-19T02:04:08.031 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.032 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.033 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.035 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.036 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-19T02:04:08.036 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.036 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.036 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.037 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.038 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.039 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.040 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.041 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-19T02:04:08.042 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:08.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-19T02:04:08.044 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-19T02:04:08.045 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-19T02:04:08.046 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 189423, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 51, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-19T02:04:08.047 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7528, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 158, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 8464, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 13, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-19T02:04:08.048 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 1590, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 1590, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1590, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.673988682, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000423892 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1590, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1591 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.049 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 1590, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 1590, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 792, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 796, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-03-19T02:04:08.050 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-19T02:04:08.051 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-19T02:04:08.052 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 87, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 2, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 5, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 5, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 0, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-03-19T02:04:08.053 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-19T02:04:08.054 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 1834, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1113977, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 1834, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1834, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1113977, 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:08.055 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 1590, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 593, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1590, 2024-03-19T02:04:08.056 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 593, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:08.057 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 1590, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 1590, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1590, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1590, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-19T02:04:08.058 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-19T02:04:08.059 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x5641d2b0a6e0": { 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-19T02:04:08.060 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-19T02:04:08.061 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:08.061 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-19T02:04:08.061 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-19T02:04:08.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:08 smithi144 ceph-mon[67199]: pgmap v691: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T02:04:08.887 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-19T02:04:08.888 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-19T02:04:08.888 DEBUG:tasks.check_counter:Getting stats from c 2024-03-19T02:04:08.888 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-19T02:04:09.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:08 smithi092 ceph-mon[62574]: pgmap v691: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T02:04:09.202 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T02:04:09.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:08 smithi159 ceph-mon[67110]: pgmap v691: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-19T02:04:10.952 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:10 smithi159 ceph-mon[67110]: pgmap v692: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-19T02:04:11.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:10 smithi092 ceph-mon[62574]: pgmap v692: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-19T02:04:11.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:10 smithi144 ceph-mon[67199]: pgmap v692: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-19T02:04:11.127 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 3321, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 5717, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 972812, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 4429387, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 13, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.253519947, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.109432322, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.106663811, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.037846971, 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:11.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5715, 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.093979354, 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000016444 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4069, 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000600963, 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000147 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 488672, 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 1460912 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 896, 2024-03-19T02:04:11.129 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 882, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 986080, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 354683, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.068198319, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.022801960, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.033179354, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.015946755, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 881, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.024485992, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000027793 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.130 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 827408, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 142384 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 454, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 791, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 142819, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 628849, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-03-19T02:04:11.131 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.049056883, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.025157601, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.019875096, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.003063294, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 790, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.023739954, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000030050 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1029, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000177894, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000172 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 375600 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4490, 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.906747711, 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000201948 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.133 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 14, 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000242186, 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000017299 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 602, 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 602, 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 602, 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.058947458, 2024-03-19T02:04:11.134 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003420178 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 15, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 1, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 14, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 13, 2024-03-19T02:04:11.135 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 4, 2024-03-19T02:04:11.136 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 1, 2024-03-19T02:04:11.136 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 1, 2024-03-19T02:04:11.136 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 3, 2024-03-19T02:04:11.136 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 606, 2024-03-19T02:04:11.136 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 602, 2024-03-19T02:04:11.136 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-03-19T02:04:11.136 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 1, 2024-03-19T02:04:11.136 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 1, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 1, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 1, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-03-19T02:04:11.137 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 599, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 601, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 598, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:04:11.138 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:04:11.139 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 701, 2024-03-19T02:04:11.139 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 699, 2024-03-19T02:04:11.139 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 2, 2024-03-19T02:04:11.139 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 601 2024-03-19T02:04:11.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.139 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-19T02:04:11.139 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-19T02:04:11.139 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 2, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 1, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 1, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-19T02:04:11.140 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 1, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 1969, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-03-19T02:04:11.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 1969, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 3, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 3, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 6485062, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1905, 2024-03-19T02:04:11.142 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.946956688, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001546958 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 19, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 22, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 3, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 18, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 21, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 3, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 15, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 21, 2024-03-19T02:04:11.143 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 6, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 1, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 3, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 2, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 51948, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223612 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 1804, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 1, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 602, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 9, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 5, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000300826, 2024-03-19T02:04:11.144 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000300826 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.145 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-19T02:04:11.146 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-19T02:04:11.148 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000080490, 2024-03-19T02:04:11.148 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000080490 2024-03-19T02:04:11.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.148 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-19T02:04:11.148 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.148 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.148 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.150 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-19T02:04:11.150 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.150 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-19T02:04:11.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001574820, 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001574820 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-19T02:04:11.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.153 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 598, 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.054590344, 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003435769 2024-03-19T02:04:11.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002400978, 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002400978 2024-03-19T02:04:11.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 1, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 0, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-03-19T02:04:11.157 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 87, 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 87, 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 674, 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:04:11.158 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:04:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-19T02:04:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 271135, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 105, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-19T02:04:11.161 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 7528, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 158, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 76853, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 318, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-19T02:04:11.162 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 2596, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 2306250, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 32, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 2596, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2596, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.983925260, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001534639 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2596, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2624 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.163 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 2596, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 1, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 2595, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 1, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 1906, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 49, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 13, 2024-03-19T02:04:11.164 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 598, 2024-03-19T02:04:11.165 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-19T02:04:11.166 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 87, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 2, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 9, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 9, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 23, 2024-03-19T02:04:11.170 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 4 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-19T02:04:11.171 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 1, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 1, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 1, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 1, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 4667, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1602993, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:11.172 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 4667, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4667, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1602993, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:11.173 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 2596, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 2306351, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2596, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 2306351, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:11.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 2596, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 2596, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2596, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 2596, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.175 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 1, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 101, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 1, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1, 2024-03-19T02:04:11.176 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 101, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55c7ce998500": { 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 1969, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 2290758, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 1969, 2024-03-19T02:04:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1905, 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 2290758, 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:11.178 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-19T02:04:11.932 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-03-19T02:04:11.933 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-03-19T02:04:11.933 DEBUG:tasks.check_counter:Getting stats from f 2024-03-19T02:04:11.933 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-19T02:04:12.242 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T02:04:12.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:12 smithi159 ceph-mon[67110]: pgmap v693: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T02:04:13.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:12 smithi092 ceph-mon[62574]: pgmap v693: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T02:04:13.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:12 smithi144 ceph-mon[67199]: pgmap v693: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T02:04:14.144 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-19T02:04:14.144 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:04:14.144 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 21, 2024-03-19T02:04:14.144 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 23, 2024-03-19T02:04:14.144 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 5221, 2024-03-19T02:04:14.144 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 55461, 2024-03-19T02:04:14.145 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 12, 2024-03-19T02:04:14.145 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2024-03-19T02:04:14.145 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.003984516, 2024-03-19T02:04:14.145 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.000258823, 2024-03-19T02:04:14.145 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.002274736, 2024-03-19T02:04:14.145 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000333012, 2024-03-19T02:04:14.145 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:14.145 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 21, 2024-03-19T02:04:14.145 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.011370847, 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000541468 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4, 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000297, 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000074 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.146 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 13, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 181, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 3451, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 380999, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.007934041, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.004911473, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.001688165, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000261811, 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:14.147 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 179, 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.015774576, 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000088126 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 375328 2024-03-19T02:04:14.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 469, 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 706, 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 867199, 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 226570, 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 5, 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.057560541, 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.022796376, 2024-03-19T02:04:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.032390793, 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.006817115, 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 706, 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.028506008, 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000040376 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 610, 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000118631, 2024-03-19T02:04:14.150 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000194 2024-03-19T02:04:14.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.151 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 825664, 2024-03-19T02:04:14.151 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 143456 2024-03-19T02:04:14.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.151 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-19T02:04:14.151 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-19T02:04:14.151 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-19T02:04:14.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.151 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 41, 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000964387, 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000023521 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-19T02:04:14.152 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000004571, 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000002285 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 10, 2024-03-19T02:04:14.154 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 10, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 10, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 1, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-03-19T02:04:14.155 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-19T02:04:14.156 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:04:14.157 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-03-19T02:04:14.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 1, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-03-19T02:04:14.159 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-03-19T02:04:14.160 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 2, 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-03-19T02:04:14.161 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 2, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 2, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-03-19T02:04:14.162 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 4194892, 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001227284, 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001227284 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 13, 2024-03-19T02:04:14.163 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 13, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 0, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 11, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 12, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 1, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 10, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 10, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 0, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 48776, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223612 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.164 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.165 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.167 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.168 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.169 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-19T02:04:14.169 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.169 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.169 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.170 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.171 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.172 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-19T02:04:14.173 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.174 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.175 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.176 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 0, 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 0, 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 0, 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 0, 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2024-03-19T02:04:14.177 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 0, 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:04:14.178 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-19T02:04:14.179 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-19T02:04:14.180 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 229959, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 71, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-19T02:04:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 7528, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 158, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-19T02:04:14.182 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 50664, 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 198, 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 35, 2024-03-19T02:04:14.183 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 10710, 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 33, 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 35, 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 35, 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.187996445, 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005371327 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 35, 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 57 2024-03-19T02:04:14.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 35, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 0, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 35, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 0, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 1, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 5, 2024-03-19T02:04:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 6, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 0, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:04:14.186 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-19T02:04:14.187 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-19T02:04:14.188 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 87, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 2, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 8, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 8, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 14, 2024-03-19T02:04:14.189 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 1 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-19T02:04:14.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:04:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 499, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 814146, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 499, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 499, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 814146, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.192 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 35, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 10710, 2024-03-19T02:04:14.193 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 35, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 10710, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:14.194 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 35, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 35, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 35, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 35, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-19T02:04:14.195 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:14.196 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55bfbd864500": { 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 2, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 588, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 2, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 588, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:14.197 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:14.198 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:14.198 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:14.198 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:14.198 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-19T02:04:14.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:14 smithi159 ceph-mon[67110]: pgmap v694: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T02:04:15.048 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-19T02:04:15.048 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-19T02:04:15.048 DEBUG:tasks.check_counter:Getting stats from i 2024-03-19T02:04:15.048 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-19T02:04:15.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:14 smithi092 ceph-mon[62574]: pgmap v694: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T02:04:15.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:14 smithi144 ceph-mon[67199]: pgmap v694: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T02:04:15.352 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T02:04:16.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:16 smithi159 ceph-mon[67110]: pgmap v695: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T02:04:17.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:16 smithi092 ceph-mon[62574]: pgmap v695: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T02:04:17.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:16 smithi144 ceph-mon[67199]: pgmap v695: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1735, 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 731, 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1333285, 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 456741, 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 25, 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.153715602, 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.042672536, 2024-03-19T02:04:17.206 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.068170715, 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.041278763, 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 731, 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.168498193, 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000230503 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1575, 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000221389, 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000140 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1217152, 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 194896 2024-03-19T02:04:17.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1018, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1011, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 365894, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 785677, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 10, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.084309720, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.036993093, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.027957057, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.018085791, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1009, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.441165980, 2024-03-19T02:04:17.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000437230 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1041, 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000172943, 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000166 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 380960 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1730, 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1155, 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 400439, 2024-03-19T02:04:17.209 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 2300386, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 98, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 10, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.127514958, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.029500569, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.054244580, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.037796972, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1153, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.433645191, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000376101 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1435, 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000221475, 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000154 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 38624, 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 1904 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-19T02:04:17.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1024, 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.164525917, 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000160669 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 22, 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000418303, 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000019013 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 49, 2024-03-19T02:04:17.212 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 48, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 48, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.075053407, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001563612 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 1, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 301, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 24, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 5, 2024-03-19T02:04:17.213 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 19, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 16, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 1, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 2, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 2, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 2, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 66, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 57, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 1, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-19T02:04:17.214 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 1, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 1, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 1, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:04:17.215 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 2, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 14, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 12, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 1, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 21, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 4, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 2, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 22 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.216 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 7, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 2, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 2, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-19T02:04:17.217 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 1, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-19T02:04:17.218 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 168, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 155, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 155, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 13, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 2, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 51, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 50, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 50, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 1, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 1, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 1, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4952984, 2024-03-19T02:04:17.219 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 4987364, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 150, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.242706428, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001618042 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 23, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 26, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 3, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 18, 2024-03-19T02:04:17.220 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 20, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 2, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 24, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 28, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 4, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 2, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 17, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 15, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 51512, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 231804 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 74, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 4, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 49, 2024-03-19T02:04:17.221 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 20, 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 10, 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5, 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.004300866, 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000860173 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 18, 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.004507021, 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000250390 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.222 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.223 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000117527, 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000117527 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-19T02:04:17.224 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.226 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-19T02:04:17.226 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8, 2024-03-19T02:04:17.226 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.013480955, 2024-03-19T02:04:17.226 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001685119 2024-03-19T02:04:17.226 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000235307, 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000235307 2024-03-19T02:04:17.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000558000, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000558000 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002907400, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001453700 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.228 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.018221352, 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.009110676 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-19T02:04:17.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.007640535, 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003820267 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.230 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8, 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.023084444, 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002885555 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.231 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.232 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 3, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 2, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 4, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 4, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 681, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.233 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:04:17.234 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:04:17.235 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 295687, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 109, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:04:17.236 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 7528, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 158, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 94418, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 622, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-19T02:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 1226, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 1093330, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 37, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 1226, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1226, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 20.602607417, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.016804736 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-19T02:04:17.238 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1226, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1870 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 1226, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 2, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 1224, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 282, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 24, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 2, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 152, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 162, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-19T02:04:17.239 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 514, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 48, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-19T02:04:17.240 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 1, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-19T02:04:17.241 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 87, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 3, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 673, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-03-19T02:04:17.242 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 12 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 1, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 1, 2024-03-19T02:04:17.243 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 2, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 2, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 4479, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1636689, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 4479, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4479, 2024-03-19T02:04:17.244 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1636689, 2024-03-19T02:04:17.245 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:17.245 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.245 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.245 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.245 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:17.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.245 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 1226, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 1093532, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1226, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1093532, 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:17.246 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 1226, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 1226, 2024-03-19T02:04:17.247 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1226, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1226, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 2, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 202, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 2, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 202, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x559a67eb4500": { 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 168, 2024-03-19T02:04:17.249 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 793060, 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 168, 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 150, 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 793060, 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:17.250 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-19T02:04:17.990 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-03-19T02:04:17.990 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-03-19T02:04:17.990 DEBUG:tasks.check_counter:Getting stats from l 2024-03-19T02:04:17.990 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810 shell --fsid d1e857f4-e591-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-19T02:04:18.294 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/mon.c/config 2024-03-19T02:04:19.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:18 smithi092 ceph-mon[62574]: pgmap v696: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T02:04:19.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:18 smithi144 ceph-mon[67199]: pgmap v696: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T02:04:19.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:18 smithi159 ceph-mon[67110]: pgmap v696: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-19T02:04:20.187 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2344, 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2346, 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 664123, 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 745399, 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.137847215, 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.041723401, 2024-03-19T02:04:20.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.034730384, 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.051974935, 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2344, 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.044799770, 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000019112 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 248, 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 236, 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 816284, 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 141466, 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 3, 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.027586738, 2024-03-19T02:04:20.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.008397896, 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.019310102, 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.003981657, 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 235, 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.007381951, 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000031412 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 815728, 2024-03-19T02:04:20.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 140768 2024-03-19T02:04:20.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.192 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-19T02:04:20.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 6, 2024-03-19T02:04:20.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 176, 2024-03-19T02:04:20.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 3452, 2024-03-19T02:04:20.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 378414, 2024-03-19T02:04:20.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2024-03-19T02:04:20.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.007253756, 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.004883871, 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.001324754, 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000022493, 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 175, 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.006515903, 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000037233 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 378096 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-19T02:04:20.194 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3957, 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.079503051, 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000020091 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 0, 2024-03-19T02:04:20.197 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 0, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 0, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 1, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2024-03-19T02:04:20.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-03-19T02:04:20.199 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-03-19T02:04:20.200 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-19T02:04:20.201 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-03-19T02:04:20.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-03-19T02:04:20.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-03-19T02:04:20.204 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 6, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-03-19T02:04:20.205 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 4195956, 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 4195815, 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 6 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 3, 2024-03-19T02:04:20.206 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 3, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 0, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 1, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 1, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 0, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 0, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 0, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 0, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 47476, 2024-03-19T02:04:20.207 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 215420 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.208 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-19T02:04:20.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-19T02:04:20.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-19T02:04:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.216 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.217 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-19T02:04:20.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 0, 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 2, 2024-03-19T02:04:20.221 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 2, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 3694, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-19T02:04:20.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-19T02:04:20.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-19T02:04:20.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 181303, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 46, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-19T02:04:20.225 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 7528, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 158, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 8464, 2024-03-19T02:04:20.226 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 13, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 2347, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 21, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 3, 2024-03-19T02:04:20.227 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 2347, 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2347, 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.962981913, 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000410303 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2347, 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2348 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 2347, 2024-03-19T02:04:20.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 2347, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 1534, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 811, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-19T02:04:20.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 1, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-19T02:04:20.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-19T02:04:20.231 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-19T02:04:20.232 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 87, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 2, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 4, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 4, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2024-03-19T02:04:20.233 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-19T02:04:20.234 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 2594, 2024-03-19T02:04:20.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1234795, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 2594, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2594, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1234795, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 2347, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 1657, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2347, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1657, 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:20.237 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:20.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 2347, 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 2347, 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2347, 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 2347, 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:20.239 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.240 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x556c2c0b46e0": { 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-19T02:04:20.241 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-19T02:04:20.242 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-03-19T02:04:20.242 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-03-19T02:04:20.242 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-19T02:04:20.242 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-19T02:04:20.242 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-19T02:04:20.242 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-19T02:04:20.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:20.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-19T02:04:20.242 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-19T02:04:20.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:20 smithi159 ceph-mon[67110]: pgmap v697: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T02:04:21.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:20 smithi092 ceph-mon[62574]: pgmap v697: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T02:04:21.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:20 smithi144 ceph-mon[67199]: pgmap v697: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-19T02:04:21.114 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-19T02:04:21.114 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-19T02:04:21.114 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-19T02:04:21.124 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-19T02:04:21.125 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T02:04:21.125 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T02:04:21.141 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-19T02:04:21.142 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-19T02:04:21.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T02:04:21.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.347+0000 7fd16e275640 1 -- 172.21.15.92:0/841820058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168104c50 msgr2=0x7fd168107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.347+0000 7fd16e275640 1 --2- 172.21.15.92:0/841820058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168104c50 0x7fd168107040 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009900 tx=0x7fd15c02bc30 comp rx=0 tx=0).stop 2024-03-19T02:04:21.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.347+0000 7fd16e275640 1 -- 172.21.15.92:0/841820058 shutdown_connections 2024-03-19T02:04:21.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.347+0000 7fd16e275640 1 --2- 172.21.15.92:0/841820058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168107580 0x7fd168109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.347+0000 7fd16e275640 1 --2- 172.21.15.92:0/841820058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168104c50 0x7fd168107040 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.347+0000 7fd16e275640 1 --2- 172.21.15.92:0/841820058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd168102320 0x7fd168104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.347+0000 7fd16e275640 1 -- 172.21.15.92:0/841820058 >> 172.21.15.92:0/841820058 conn(0x7fd1680fbcb0 msgr2=0x7fd1680fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:21.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.347+0000 7fd16e275640 1 -- 172.21.15.92:0/841820058 shutdown_connections 2024-03-19T02:04:21.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.347+0000 7fd16e275640 1 -- 172.21.15.92:0/841820058 wait complete. 2024-03-19T02:04:21.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16e275640 1 Processor -- start 2024-03-19T02:04:21.349 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16e275640 1 -- start start 2024-03-19T02:04:21.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16e275640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd168102320 0x7fd168197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:21.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16e275640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168104c50 0x7fd1681980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:21.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16e275640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168107580 0x7fd16819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:21.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16e275640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810d4b0 con 0x7fd168107580 2024-03-19T02:04:21.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16e275640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810d330 con 0x7fd168104c50 2024-03-19T02:04:21.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16e275640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810d630 con 0x7fd168102320 2024-03-19T02:04:21.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16c812640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168107580 0x7fd16819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:21.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16c812640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168107580 0x7fd16819f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59122/0 (socket says 172.21.15.92:59122) 2024-03-19T02:04:21.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd16c812640 1 -- 172.21.15.92:0/3959257660 learned_addr learned my addr 172.21.15.92:0/3959257660 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.348+0000 7fd167fff640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd168102320 0x7fd168197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd1677fe640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168104c50 0x7fd1681980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd16c812640 1 -- 172.21.15.92:0/3959257660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd168102320 msgr2=0x7fd168197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd16c812640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd168102320 0x7fd168197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd16c812640 1 -- 172.21.15.92:0/3959257660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168104c50 msgr2=0x7fd1681980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd16c812640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168104c50 0x7fd1681980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd16c812640 1 -- 172.21.15.92:0/3959257660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16819f7a0 con 0x7fd168107580 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd16c812640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168107580 0x7fd16819f130 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd16000eb10 tx=0x7fd16000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd1657fa640 1 -- 172.21.15.92:0/3959257660 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160004b20 con 0x7fd168107580 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd1657fa640 1 -- 172.21.15.92:0/3959257660 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd160004cc0 con 0x7fd168107580 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16819fa90 con 0x7fd168107580 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd1657fa640 1 -- 172.21.15.92:0/3959257660 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160010760 con 0x7fd168107580 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.349+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16819ff60 con 0x7fd168107580 2024-03-19T02:04:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.350+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1680fd7a0 con 0x7fd168107580 2024-03-19T02:04:21.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.351+0000 7fd1657fa640 1 -- 172.21.15.92:0/3959257660 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fd160010900 con 0x7fd168107580 2024-03-19T02:04:21.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.352+0000 7fd1657fa640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd1500a2800 0x7fd1500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:21.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.352+0000 7fd1657fa640 1 -- 172.21.15.92:0/3959257660 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd1600cf1f0 con 0x7fd168107580 2024-03-19T02:04:21.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.352+0000 7fd167fff640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd1500a2800 0x7fd1500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:21.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.352+0000 7fd167fff640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd1500a2800 0x7fd1500a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd158009bd0 tx=0x7fd158005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:21.355 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.355+0000 7fd1657fa640 1 -- 172.21.15.92:0/3959257660 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd160053740 con 0x7fd168107580 2024-03-19T02:04:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.537+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd168062420 con 0x7fd168107580 2024-03-19T02:04:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.539+0000 7fd1657fa640 1 -- 172.21.15.92:0/3959257660 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22690 (secure 0 0 0) 0x7fd16009c2a0 con 0x7fd168107580 2024-03-19T02:04:21.539 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T02:04:21.539 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":87,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:51:40.581546+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T02:04:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd1500a2800 msgr2=0x7fd1500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd1500a2800 0x7fd1500a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd158009bd0 tx=0x7fd158005f90 comp rx=0 tx=0).stop 2024-03-19T02:04:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168107580 msgr2=0x7fd16819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168107580 0x7fd16819f130 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd16000eb10 tx=0x7fd16000c2d0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 shutdown_connections 2024-03-19T02:04:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd168107580 0x7fd16819f130 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd168104c50 0x7fd1681980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fd1500a2800 0x7fd1500a4cc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 --2- 172.21.15.92:0/3959257660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd168102320 0x7fd168197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 >> 172.21.15.92:0/3959257660 conn(0x7fd1680fbcb0 msgr2=0x7fd168102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 shutdown_connections 2024-03-19T02:04:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.541+0000 7fd16e275640 1 -- 172.21.15.92:0/3959257660 wait complete. 2024-03-19T02:04:21.553 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-19T01:46:07.695753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-19T01:50:47.888441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-19T01:50:49.068308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-19T02:04:21.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-19T02:04:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.661+0000 7f796abc4640 1 -- 172.21.15.92:0/629110379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964104c90 msgr2=0x7f7964107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.661+0000 7f796abc4640 1 --2- 172.21.15.92:0/629110379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964104c90 0x7f7964107080 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7958009900 tx=0x7f795802bc30 comp rx=0 tx=0).stop 2024-03-19T02:04:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.661+0000 7f796abc4640 1 -- 172.21.15.92:0/629110379 shutdown_connections 2024-03-19T02:04:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.661+0000 7f796abc4640 1 --2- 172.21.15.92:0/629110379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79641075c0 0x7f79641099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.661+0000 7f796abc4640 1 --2- 172.21.15.92:0/629110379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964104c90 0x7f7964107080 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.661+0000 7f796abc4640 1 --2- 172.21.15.92:0/629110379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7964102360 0x7f7964104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.661+0000 7f796abc4640 1 -- 172.21.15.92:0/629110379 >> 172.21.15.92:0/629110379 conn(0x7f79640fbcb0 msgr2=0x7f79640fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.661+0000 7f796abc4640 1 -- 172.21.15.92:0/629110379 shutdown_connections 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.661+0000 7f796abc4640 1 -- 172.21.15.92:0/629110379 wait complete. 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f796abc4640 1 Processor -- start 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f796abc4640 1 -- start start 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f796abc4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7964102360 0x7f7964197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f796abc4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964104c90 0x7f79641980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f796abc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79641075c0 0x7f796419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f796abc4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f796410d4b0 con 0x7f7964102360 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f796abc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f796410d330 con 0x7f79641075c0 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f796abc4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f796410d630 con 0x7f7964104c90 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f7968960640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7964102360 0x7f7964197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f7968960640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7964102360 0x7f7964197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59138/0 (socket says 172.21.15.92:59138) 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f7968960640 1 -- 172.21.15.92:0/234771123 learned_addr learned my addr 172.21.15.92:0/234771123 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f7963fff640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964104c90 0x7f79641980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.662+0000 7f7969161640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79641075c0 0x7f796419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f7968960640 1 -- 172.21.15.92:0/234771123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964104c90 msgr2=0x7f79641980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f7968960640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964104c90 0x7f79641980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f7968960640 1 -- 172.21.15.92:0/234771123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79641075c0 msgr2=0x7f796419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f7968960640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79641075c0 0x7f796419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f7968960640 1 -- 172.21.15.92:0/234771123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796419f7d0 con 0x7f7964102360 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f7968960640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7964102360 0x7f7964197bb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f795400e9f0 tx=0x7f795400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f7961ffb640 1 -- 172.21.15.92:0/234771123 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795400cd80 con 0x7f7964102360 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796419fac0 con 0x7f7964102360 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f7961ffb640 1 -- 172.21.15.92:0/234771123 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7954010910 con 0x7f7964102360 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f7961ffb640 1 -- 172.21.15.92:0/234771123 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540188f0 con 0x7f7964102360 2024-03-19T02:04:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.663+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79641a0030 con 0x7f7964102360 2024-03-19T02:04:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.664+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79640fd7a0 con 0x7f7964102360 2024-03-19T02:04:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.665+0000 7f7961ffb640 1 -- 172.21.15.92:0/234771123 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f7954010ac0 con 0x7f7964102360 2024-03-19T02:04:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.666+0000 7f7961ffb640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f794c0a2800 0x7f794c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:21.667 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.666+0000 7f7963fff640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f794c0a2800 0x7f794c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:21.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.666+0000 7f7961ffb640 1 -- 172.21.15.92:0/234771123 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f79540cf1c0 con 0x7f7964102360 2024-03-19T02:04:21.668 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.667+0000 7f7963fff640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f794c0a2800 0x7f794c0a4cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7958000c00 tx=0x7f7958005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:21.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.670+0000 7f7961ffb640 1 -- 172.21.15.92:0/234771123 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f795409bc20 con 0x7f7964102360 2024-03-19T02:04:21.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:21 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3959257660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T02:04:21.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.880+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f796402d1f0 con 0x7f7964102360 2024-03-19T02:04:21.880 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.880+0000 7f7961ffb640 1 -- 172.21.15.92:0/234771123 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f79540860a0 con 0x7f7964102360 2024-03-19T02:04:21.880 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2024-03-19T02:04:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f794c0a2800 msgr2=0x7f794c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f794c0a2800 0x7f794c0a4cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7958000c00 tx=0x7f7958005fd0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7964102360 msgr2=0x7f7964197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7964102360 0x7f7964197bb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f795400e9f0 tx=0x7f795400eeb0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 shutdown_connections 2024-03-19T02:04:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79641075c0 0x7f796419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f794c0a2800 0x7f794c0a4cc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7964104c90 0x7f79641980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 --2- 172.21.15.92:0/234771123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7964102360 0x7f7964197bb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 >> 172.21.15.92:0/234771123 conn(0x7f79640fbcb0 msgr2=0x7f7964102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 shutdown_connections 2024-03-19T02:04:21.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.882+0000 7f796abc4640 1 -- 172.21.15.92:0/234771123 wait complete. 2024-03-19T02:04:21.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-19T02:04:22.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 -- 172.21.15.92:0/3490560334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1380ffb00 msgr2=0x7ff1380fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/3490560334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1380ffb00 0x7ff1380fff20 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff130009900 tx=0x7ff13002bc80 comp rx=0 tx=0).stop 2024-03-19T02:04:22.001 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 -- 172.21.15.92:0/3490560334 shutdown_connections 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/3490560334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff138103530 0x7ff138109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/3490560334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff138102b70 0x7ff138102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/3490560334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1380ffb00 0x7ff1380fff20 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 -- 172.21.15.92:0/3490560334 >> 172.21.15.92:0/3490560334 conn(0x7ff1380fb450 msgr2=0x7ff1380fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 -- 172.21.15.92:0/3490560334 shutdown_connections 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 -- 172.21.15.92:0/3490560334 wait complete. 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 Processor -- start 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 -- start start 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1380ffb00 0x7ff138197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff138102b70 0x7ff1381980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.002 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:21.999+0000 7ff13d4a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff138103530 0x7ff13819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff13d4a1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff13810d4d0 con 0x7ff138103530 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff13d4a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff13810d350 con 0x7ff1380ffb00 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff13d4a1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff13810d650 con 0x7ff138102b70 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff138103530 0x7ff13819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1377fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff138103530 0x7ff13819f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59150/0 (socket says 172.21.15.92:59150) 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1367fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff138102b70 0x7ff1381980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1377fe640 1 -- 172.21.15.92:0/1173507104 learned_addr learned my addr 172.21.15.92:0/1173507104 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff136ffd640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1380ffb00 0x7ff138197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1377fe640 1 -- 172.21.15.92:0/1173507104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff138102b70 msgr2=0x7ff1381980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1377fe640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff138102b70 0x7ff1381980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1377fe640 1 -- 172.21.15.92:0/1173507104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1380ffb00 msgr2=0x7ff138197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.003 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1377fe640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1380ffb00 0x7ff138197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1377fe640 1 -- 172.21.15.92:0/1173507104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13819f7b0 con 0x7ff138103530 2024-03-19T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff1377fe640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff138103530 0x7ff13819f140 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff12800eb10 tx=0x7ff12800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff12ffff640 1 -- 172.21.15.92:0/1173507104 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128004b20 con 0x7ff138103530 2024-03-19T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13819faa0 con 0x7ff138103530 2024-03-19T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff12ffff640 1 -- 172.21.15.92:0/1173507104 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff128004cc0 con 0x7ff138103530 2024-03-19T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff12ffff640 1 -- 172.21.15.92:0/1173507104 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128010760 con 0x7ff138103530 2024-03-19T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.000+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13819ffa0 con 0x7ff138103530 2024-03-19T02:04:22.004 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.001+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff138108d70 con 0x7ff138103530 2024-03-19T02:04:22.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.003+0000 7ff12ffff640 1 -- 172.21.15.92:0/1173507104 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff128005000 con 0x7ff138103530 2024-03-19T02:04:22.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.003+0000 7ff12ffff640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff1240a2800 0x7ff1240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.003+0000 7ff136ffd640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff1240a2800 0x7ff1240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.004+0000 7ff12ffff640 1 -- 172.21.15.92:0/1173507104 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff1280cf1f0 con 0x7ff138103530 2024-03-19T02:04:22.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.004+0000 7ff136ffd640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff1240a2800 0x7ff1240a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff130000c00 tx=0x7ff13002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:22.007 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.007+0000 7ff12ffff640 1 -- 172.21.15.92:0/1173507104 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff128053740 con 0x7ff138103530 2024-03-19T02:04:22.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:21 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3959257660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T02:04:22.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.186+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff13802d440 con 0x7ff138103530 2024-03-19T02:04:22.187 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.187+0000 7ff12ffff640 1 -- 172.21.15.92:0/1173507104 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7ff12809c2a0 con 0x7ff138103530 2024-03-19T02:04:22.187 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff1240a2800 msgr2=0x7ff1240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff1240a2800 0x7ff1240a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff130000c00 tx=0x7ff13002df90 comp rx=0 tx=0).stop 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff138103530 msgr2=0x7ff13819f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff138103530 0x7ff13819f140 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff12800eb10 tx=0x7ff12800c2d0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 shutdown_connections 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff138103530 0x7ff13819f140 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7ff1240a2800 0x7ff1240a4cc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff138102b70 0x7ff1381980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 --2- 172.21.15.92:0/1173507104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1380ffb00 0x7ff138197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 >> 172.21.15.92:0/1173507104 conn(0x7ff1380fb450 msgr2=0x7ff138105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:22.190 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 shutdown_connections 2024-03-19T02:04:22.191 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.189+0000 7ff13d4a1640 1 -- 172.21.15.92:0/1173507104 wait complete. 2024-03-19T02:04:22.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-19T02:04:22.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:21 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3959257660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.306+0000 7fdd93f10640 1 -- 172.21.15.92:0/1215432732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c100720 msgr2=0x7fdd8c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.306+0000 7fdd93f10640 1 --2- 172.21.15.92:0/1215432732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c100720 0x7fdd8c100b40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009900 tx=0x7fdd8802bc30 comp rx=0 tx=0).stop 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.306+0000 7fdd93f10640 1 -- 172.21.15.92:0/1215432732 shutdown_connections 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.306+0000 7fdd93f10640 1 --2- 172.21.15.92:0/1215432732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd8c1022d0 0x7fdd8c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.306+0000 7fdd93f10640 1 --2- 172.21.15.92:0/1215432732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c101910 0x7fdd8c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.306+0000 7fdd93f10640 1 --2- 172.21.15.92:0/1215432732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c100720 0x7fdd8c100b40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.306+0000 7fdd93f10640 1 -- 172.21.15.92:0/1215432732 >> 172.21.15.92:0/1215432732 conn(0x7fdd8c0fbcb0 msgr2=0x7fdd8c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.306+0000 7fdd93f10640 1 -- 172.21.15.92:0/1215432732 shutdown_connections 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.306+0000 7fdd93f10640 1 -- 172.21.15.92:0/1215432732 wait complete. 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd93f10640 1 Processor -- start 2024-03-19T02:04:22.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd93f10640 1 -- start start 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd93f10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c100720 0x7fdd8c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd93f10640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd8c101910 0x7fdd8c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd93f10640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c1022d0 0x7fdd8c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd93f10640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10c370 con 0x7fdd8c1022d0 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd93f10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10c1f0 con 0x7fdd8c100720 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd93f10640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10c4f0 con 0x7fdd8c101910 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd924ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c1022d0 0x7fdd8c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd924ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c1022d0 0x7fdd8c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59170/0 (socket says 172.21.15.92:59170) 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd924ad640 1 -- 172.21.15.92:0/3345862914 learned_addr learned my addr 172.21.15.92:0/3345862914 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd914ab640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd8c101910 0x7fdd8c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd91cac640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c100720 0x7fdd8c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd924ad640 1 -- 172.21.15.92:0/3345862914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd8c101910 msgr2=0x7fdd8c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd924ad640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd8c101910 0x7fdd8c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd924ad640 1 -- 172.21.15.92:0/3345862914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c100720 msgr2=0x7fdd8c0694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.307+0000 7fdd924ad640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c100720 0x7fdd8c0694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.308+0000 7fdd924ad640 1 -- 172.21.15.92:0/3345862914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c19f820 con 0x7fdd8c1022d0 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.308+0000 7fdd924ad640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c1022d0 0x7fdd8c19ef10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400efd0 tx=0x7fdd8400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.308+0000 7fdd82ffd640 1 -- 172.21.15.92:0/3345862914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84003b70 con 0x7fdd8c1022d0 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.308+0000 7fdd82ffd640 1 -- 172.21.15.92:0/3345862914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd84003d10 con 0x7fdd8c1022d0 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.308+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8c19fab0 con 0x7fdd8c1022d0 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.308+0000 7fdd82ffd640 1 -- 172.21.15.92:0/3345862914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840109f0 con 0x7fdd8c1022d0 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.308+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8c19ffb0 con 0x7fdd8c1022d0 2024-03-19T02:04:22.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.309+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8c107b60 con 0x7fdd8c1022d0 2024-03-19T02:04:22.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.310+0000 7fdd82ffd640 1 -- 172.21.15.92:0/3345862914 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fdd84004020 con 0x7fdd8c1022d0 2024-03-19T02:04:22.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.311+0000 7fdd82ffd640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdd780a2800 0x7fdd780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.311+0000 7fdd91cac640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdd780a2800 0x7fdd780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.311+0000 7fdd82ffd640 1 -- 172.21.15.92:0/3345862914 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdd840ceec0 con 0x7fdd8c1022d0 2024-03-19T02:04:22.312 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.311+0000 7fdd91cac640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdd780a2800 0x7fdd780a4cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdd88000c00 tx=0x7fdd88005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:22.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.315+0000 7fdd82ffd640 1 -- 172.21.15.92:0/3345862914 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdd8409bf70 con 0x7fdd8c1022d0 2024-03-19T02:04:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.499+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdd8c062420 con 0x7fdd8c1022d0 2024-03-19T02:04:22.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.500+0000 7fdd82ffd640 1 -- 172.21.15.92:0/3345862914 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fdd8409b870 con 0x7fdd8c1022d0 2024-03-19T02:04:22.500 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2024-03-19T02:04:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdd780a2800 msgr2=0x7fdd780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdd780a2800 0x7fdd780a4cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdd88000c00 tx=0x7fdd88005fd0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c1022d0 msgr2=0x7fdd8c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c1022d0 0x7fdd8c19ef10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400efd0 tx=0x7fdd8400c6a0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 shutdown_connections 2024-03-19T02:04:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd8c1022d0 0x7fdd8c19ef10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fdd780a2800 0x7fdd780a4cc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd8c101910 0x7fdd8c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 --2- 172.21.15.92:0/3345862914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd8c100720 0x7fdd8c0694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 >> 172.21.15.92:0/3345862914 conn(0x7fdd8c0fbcb0 msgr2=0x7fdd8c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 shutdown_connections 2024-03-19T02:04:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.502+0000 7fdd93f10640 1 -- 172.21.15.92:0/3345862914 wait complete. 2024-03-19T02:04:22.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-19T02:04:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.617+0000 7f4052bb9640 1 -- 172.21.15.92:0/4139266699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 msgr2=0x7f404c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.617+0000 7f4052bb9640 1 --2- 172.21.15.92:0/4139266699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 0x7f404c101db0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4038009900 tx=0x7f403802bc30 comp rx=0 tx=0).stop 2024-03-19T02:04:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.617+0000 7f4052bb9640 1 -- 172.21.15.92:0/4139266699 shutdown_connections 2024-03-19T02:04:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.617+0000 7f4052bb9640 1 --2- 172.21.15.92:0/4139266699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f404c1022f0 0x7f404c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.617+0000 7f4052bb9640 1 --2- 172.21.15.92:0/4139266699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 0x7f404c101db0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.617+0000 7f4052bb9640 1 --2- 172.21.15.92:0/4139266699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404c100740 0x7f404c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.617+0000 7f4052bb9640 1 -- 172.21.15.92:0/4139266699 >> 172.21.15.92:0/4139266699 conn(0x7f404c0fbcd0 msgr2=0x7f404c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.617+0000 7f4052bb9640 1 -- 172.21.15.92:0/4139266699 shutdown_connections 2024-03-19T02:04:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.617+0000 7f4052bb9640 1 -- 172.21.15.92:0/4139266699 wait complete. 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4052bb9640 1 Processor -- start 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4052bb9640 1 -- start start 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4052bb9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404c100740 0x7f404c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4052bb9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 0x7f404c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4052bb9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f404c1022f0 0x7f404c1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4052bb9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f404c10c3a0 con 0x7f404c101930 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4052bb9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f404c10c220 con 0x7f404c100740 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4052bb9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f404c10c520 con 0x7f404c1022f0 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f404bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 0x7f404c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f404bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 0x7f404c19ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59198/0 (socket says 172.21.15.92:59198) 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f404bfff640 1 -- 172.21.15.92:0/2561976770 learned_addr learned my addr 172.21.15.92:0/2561976770 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4050955640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404c100740 0x7f404c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f4051156640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f404c1022f0 0x7f404c1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f404bfff640 1 -- 172.21.15.92:0/2561976770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f404c1022f0 msgr2=0x7f404c1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.618+0000 7f404bfff640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f404c1022f0 0x7f404c1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.619+0000 7f404bfff640 1 -- 172.21.15.92:0/2561976770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404c100740 msgr2=0x7f404c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.619+0000 7f404bfff640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404c100740 0x7f404c06a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.619+0000 7f404bfff640 1 -- 172.21.15.92:0/2561976770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c1a19d0 con 0x7f404c101930 2024-03-19T02:04:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.619+0000 7f404bfff640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 0x7f404c19ec90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4038009230 tx=0x7f403802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.619+0000 7f4049ffb640 1 -- 172.21.15.92:0/2561976770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038041070 con 0x7f404c101930 2024-03-19T02:04:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.619+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c1a1c00 con 0x7f404c101930 2024-03-19T02:04:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.619+0000 7f4049ffb640 1 -- 172.21.15.92:0/2561976770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4038004450 con 0x7f404c101930 2024-03-19T02:04:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.619+0000 7f4049ffb640 1 -- 172.21.15.92:0/2561976770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038003e20 con 0x7f404c101930 2024-03-19T02:04:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.619+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c1a2140 con 0x7f404c101930 2024-03-19T02:04:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.620+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c107a60 con 0x7f404c101930 2024-03-19T02:04:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.622+0000 7f4049ffb640 1 -- 172.21.15.92:0/2561976770 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f4038044070 con 0x7f404c101930 2024-03-19T02:04:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.622+0000 7f4049ffb640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f403c0a2830 0x7f403c0a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.623+0000 7f4049ffb640 1 -- 172.21.15.92:0/2561976770 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f40380f0960 con 0x7f404c101930 2024-03-19T02:04:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.623+0000 7f4050955640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f403c0a2830 0x7f403c0a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.623+0000 7f4050955640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f403c0a2830 0x7f403c0a4cf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f404c101790 tx=0x7f4040005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.625+0000 7f4049ffb640 1 -- 172.21.15.92:0/2561976770 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f40380bd360 con 0x7f404c101930 2024-03-19T02:04:22.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:22 smithi092 ceph-mon[62574]: pgmap v698: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T02:04:22.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:22 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/234771123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T02:04:22.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:22 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1173507104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T02:04:22.810 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:22 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3345862914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T02:04:22.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.824+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f404c02d3d0 con 0x7f404c101930 2024-03-19T02:04:22.825 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.824+0000 7f4049ffb640 1 -- 172.21.15.92:0/2561976770 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+591 (secure 0 0 0) 0x7f40380c1400 con 0x7f404c101930 2024-03-19T02:04:22.825 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/1888848283 2024-03-20T01:43:39.363114+0000 2024-03-19T02:04:22.826 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/1866676846 2024-03-20T01:43:39.363114+0000 2024-03-19T02:04:22.826 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/1866676846 2024-03-20T01:43:39.363114+0000 2024-03-19T02:04:22.826 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6800/1465441882 2024-03-20T01:43:12.221209+0000 2024-03-19T02:04:22.826 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/239005224 2024-03-20T01:43:12.221209+0000 2024-03-19T02:04:22.826 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/2465427523 2024-03-20T01:43:39.363114+0000 2024-03-19T02:04:22.826 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6801/1465441882 2024-03-20T01:43:12.221209+0000 2024-03-19T02:04:22.827 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3016291750 2024-03-20T01:43:12.221209+0000 2024-03-19T02:04:22.827 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3316636845 2024-03-20T01:43:39.363114+0000 2024-03-19T02:04:22.827 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:0/3619099089 2024-03-20T01:43:12.221209+0000 2024-03-19T02:04:22.829 INFO:teuthology.orchestra.run.smithi092.stderr:listed 10 entries 2024-03-19T02:04:22.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f403c0a2830 msgr2=0x7f403c0a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f403c0a2830 0x7f403c0a4cf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f404c101790 tx=0x7f4040005f90 comp rx=0 tx=0).stop 2024-03-19T02:04:22.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 msgr2=0x7f404c19ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:22.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 0x7f404c19ec90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4038009230 tx=0x7f403802c650 comp rx=0 tx=0).stop 2024-03-19T02:04:22.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 shutdown_connections 2024-03-19T02:04:22.829 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f403c0a2830 0x7f403c0a4cf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f404c1022f0 0x7f404c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f404c101930 0x7f404c19ec90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 --2- 172.21.15.92:0/2561976770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404c100740 0x7f404c06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 >> 172.21.15.92:0/2561976770 conn(0x7f404c0fbcd0 msgr2=0x7f404c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 shutdown_connections 2024-03-19T02:04:22.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:22.826+0000 7f4052bb9640 1 -- 172.21.15.92:0/2561976770 wait complete. 2024-03-19T02:04:22.838 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-19T02:04:22.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T02:04:22.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-19T02:04:22.883 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2024-03-19T02:04:22.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T02:04:22.883 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-19T02:04:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:22 smithi159 ceph-mon[67110]: pgmap v698: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T02:04:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:22 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/234771123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T02:04:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:22 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1173507104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T02:04:22.955 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:22 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3345862914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T02:04:22.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T02:04:22.958 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-03-19T02:04:23.012 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-19T02:04:23.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T02:04:23.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-19T02:04:23.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-19T02:04:23.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-03-19T02:04:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:22 smithi144 ceph-mon[67199]: pgmap v698: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-19T02:04:23.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:22 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/234771123' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-19T02:04:23.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:22 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1173507104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-19T02:04:23.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:22 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3345862914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-19T02:04:23.188 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-19T02:04:23.198 INFO:tasks.cephadm:Teardown begin 2024-03-19T02:04:23.198 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-19T02:04:23.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T02:04:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 -- 172.21.15.92:0/2059345905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a74101930 msgr2=0x7f1a74101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/2059345905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a74101930 0x7f1a74101db0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009900 tx=0x7f1a5c02bc30 comp rx=0 tx=0).stop 2024-03-19T02:04:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 -- 172.21.15.92:0/2059345905 shutdown_connections 2024-03-19T02:04:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/2059345905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a741022f0 0x7f1a74108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/2059345905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a74101930 0x7f1a74101db0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/2059345905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a74100740 0x7f1a74100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 -- 172.21.15.92:0/2059345905 >> 172.21.15.92:0/2059345905 conn(0x7f1a740fbcb0 msgr2=0x7f1a740fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 -- 172.21.15.92:0/2059345905 shutdown_connections 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 -- 172.21.15.92:0/2059345905 wait complete. 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 Processor -- start 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.305+0000 7f1a79dc4640 1 -- start start 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a79dc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a74100740 0x7f1a74197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a79dc4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a74101930 0x7f1a741980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a79dc4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a741022f0 0x7f1a7419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a79dc4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7410c3f0 con 0x7f1a741022f0 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a79dc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7410c270 con 0x7f1a74100740 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a79dc4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7410c570 con 0x7f1a74101930 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a73fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a741022f0 0x7f1a7419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a73fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a741022f0 0x7f1a7419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59210/0 (socket says 172.21.15.92:59210) 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a73fff640 1 -- 172.21.15.92:0/3751415025 learned_addr learned my addr 172.21.15.92:0/3751415025 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a737fe640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a74100740 0x7f1a74197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a72ffd640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a74101930 0x7f1a741980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a73fff640 1 -- 172.21.15.92:0/3751415025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a74101930 msgr2=0x7f1a741980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a73fff640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a74101930 0x7f1a741980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a73fff640 1 -- 172.21.15.92:0/3751415025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a74100740 msgr2=0x7f1a74197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a73fff640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a74100740 0x7f1a74197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a73fff640 1 -- 172.21.15.92:0/3751415025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7419f7b0 con 0x7f1a741022f0 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a73fff640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a741022f0 0x7f1a7419f140 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00d950 tx=0x7f1a6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a70ff9640 1 -- 172.21.15.92:0/3751415025 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c017030 con 0x7f1a741022f0 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7419faa0 con 0x7f1a741022f0 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a70ff9640 1 -- 172.21.15.92:0/3751415025 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a6c004020 con 0x7f1a741022f0 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.306+0000 7f1a70ff9640 1 -- 172.21.15.92:0/3751415025 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c003c00 con 0x7f1a741022f0 2024-03-19T02:04:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.307+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7419ffc0 con 0x7f1a741022f0 2024-03-19T02:04:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.308+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a741a0250 con 0x7f1a741022f0 2024-03-19T02:04:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.309+0000 7f1a70ff9640 1 -- 172.21.15.92:0/3751415025 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f1a6c01a070 con 0x7f1a741022f0 2024-03-19T02:04:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.309+0000 7f1a70ff9640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1a600a2800 0x7f1a600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.309+0000 7f1a70ff9640 1 -- 172.21.15.92:0/3751415025 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1a6c0ce2a0 con 0x7f1a741022f0 2024-03-19T02:04:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.309+0000 7f1a737fe640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1a600a2800 0x7f1a600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.310+0000 7f1a737fe640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1a600a2800 0x7f1a600a4cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a74101790 tx=0x7f1a64009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.314+0000 7f1a70ff9640 1 -- 172.21.15.92:0/3751415025 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1a6c021070 con 0x7f1a741022f0 2024-03-19T02:04:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.557+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a741a0470 con 0x7f1a741022f0 2024-03-19T02:04:23.558 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.558+0000 7f1a70ff9640 1 -- 172.21.15.92:0/3751415025 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f1a6c09b350 con 0x7f1a741022f0 2024-03-19T02:04:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T02:04:23.558 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:50:49.881618+0000","modified":"2024-03-19T02:03:44.776081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24476,"mds_2":24475,"mds_3":24457,"mds_4":14601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6839/380126062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":380126062},{"type":"v1","addr":"172.21.15.144:6839","nonce":380126062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24476,24493,24475,24457,14601]},"id":1}]} 2024-03-19T02:04:23.559 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 30 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1a600a2800 msgr2=0x7f1a600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1a600a2800 0x7f1a600a4cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1a74101790 tx=0x7f1a64009670 comp rx=0 tx=0).stop 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a741022f0 msgr2=0x7f1a7419f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a741022f0 0x7f1a7419f140 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00d950 tx=0x7f1a6c00de10 comp rx=0 tx=0).stop 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 shutdown_connections 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a741022f0 0x7f1a7419f140 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f1a600a2800 0x7f1a600a4cc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a74101930 0x7f1a741980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 --2- 172.21.15.92:0/3751415025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a74100740 0x7f1a74197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.560+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 >> 172.21.15.92:0/3751415025 conn(0x7f1a740fbcb0 msgr2=0x7f1a740fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:23.561 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.561+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 shutdown_connections 2024-03-19T02:04:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.561+0000 7f1a79dc4640 1 -- 172.21.15.92:0/3751415025 wait complete. 2024-03-19T02:04:23.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-19T02:04:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.689+0000 7f28df997640 1 -- 172.21.15.92:0/508073874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d8100740 msgr2=0x7f28d8100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.689+0000 7f28df997640 1 --2- 172.21.15.92:0/508073874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d8100740 0x7f28d8100b60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009900 tx=0x7f28d402bc30 comp rx=0 tx=0).stop 2024-03-19T02:04:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.690+0000 7f28df997640 1 -- 172.21.15.92:0/508073874 shutdown_connections 2024-03-19T02:04:23.690 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.690+0000 7f28df997640 1 --2- 172.21.15.92:0/508073874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28d81022f0 0x7f28d8108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.690+0000 7f28df997640 1 --2- 172.21.15.92:0/508073874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28d8101930 0x7f28d8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.690+0000 7f28df997640 1 --2- 172.21.15.92:0/508073874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d8100740 0x7f28d8100b60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.690+0000 7f28df997640 1 -- 172.21.15.92:0/508073874 >> 172.21.15.92:0/508073874 conn(0x7f28d80fbcb0 msgr2=0x7f28d80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:23.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.690+0000 7f28df997640 1 -- 172.21.15.92:0/508073874 shutdown_connections 2024-03-19T02:04:23.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.690+0000 7f28df997640 1 -- 172.21.15.92:0/508073874 wait complete. 2024-03-19T02:04:23.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.690+0000 7f28df997640 1 Processor -- start 2024-03-19T02:04:23.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28df997640 1 -- start start 2024-03-19T02:04:23.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28df997640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28d8100740 0x7f28d8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:23.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28df997640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d8101930 0x7f28d8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:23.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28df997640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28d81022f0 0x7f28d819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:23.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28df997640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f28d810c3c0 con 0x7f28d8101930 2024-03-19T02:04:23.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28df997640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28d810c240 con 0x7f28d81022f0 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28df997640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f28d810c540 con 0x7f28d8100740 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28dcf32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d8101930 0x7f28d8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28ddf34640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28d81022f0 0x7f28d819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28ddf34640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28d81022f0 0x7f28d819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.92:39366/0 (socket says 172.21.15.92:39366) 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28ddf34640 1 -- 172.21.15.92:0/3664885929 learned_addr learned my addr 172.21.15.92:0/3664885929 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28dcf32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d8101930 0x7f28d8198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47652/0 (socket says 172.21.15.92:47652) 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28dd733640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28d8100740 0x7f28d8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28ddf34640 1 -- 172.21.15.92:0/3664885929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28d8100740 msgr2=0x7f28d8197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28ddf34640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28d8100740 0x7f28d8197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28ddf34640 1 -- 172.21.15.92:0/3664885929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d8101930 msgr2=0x7f28d8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28ddf34640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d8101930 0x7f28d8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.691+0000 7f28ddf34640 1 -- 172.21.15.92:0/3664885929 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d819f750 con 0x7f28d81022f0 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.692+0000 7f28ddf34640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28d81022f0 0x7f28d819f0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f28d000eb10 tx=0x7f28d000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:23.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.692+0000 7f28ce7fc640 1 -- 172.21.15.92:0/3664885929 <== mon.2 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0004b20 con 0x7f28d81022f0 2024-03-19T02:04:23.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.692+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d8102bb0 con 0x7f28d81022f0 2024-03-19T02:04:23.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.692+0000 7f28ce7fc640 1 -- 172.21.15.92:0/3664885929 <== mon.2 v2:172.21.15.144:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d0004cc0 con 0x7f28d81022f0 2024-03-19T02:04:23.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.692+0000 7f28ce7fc640 1 -- 172.21.15.92:0/3664885929 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0010760 con 0x7f28d81022f0 2024-03-19T02:04:23.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.692+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d8103100 con 0x7f28d81022f0 2024-03-19T02:04:23.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.693+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d81033c0 con 0x7f28d81022f0 2024-03-19T02:04:23.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.694+0000 7f28ce7fc640 1 -- 172.21.15.92:0/3664885929 <== mon.2 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f28d0005020 con 0x7f28d81022f0 2024-03-19T02:04:23.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.695+0000 7f28ce7fc640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f28c40a2800 0x7f28c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:23.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.695+0000 7f28dd733640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f28c40a2800 0x7f28c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:23.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.695+0000 7f28ce7fc640 1 -- 172.21.15.92:0/3664885929 <== mon.2 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f28d00cf130 con 0x7f28d81022f0 2024-03-19T02:04:23.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.696+0000 7f28dd733640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f28c40a2800 0x7f28c40a4cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f28d4000c00 tx=0x7f28d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:23.704 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.704+0000 7f28ce7fc640 1 -- 172.21.15.92:0/3664885929 <== mon.2 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f28d009c1e0 con 0x7f28d81022f0 2024-03-19T02:04:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.962+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28d8103b60 con 0x7f28d81022f0 2024-03-19T02:04:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.963+0000 7f28ce7fc640 1 -- 172.21.15.92:0/3664885929 <== mon.2 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f28d009bae0 con 0x7f28d81022f0 2024-03-19T02:04:23.963 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T02:04:23.964 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/383124452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":383124452},{"type":"v1","addr":"172.21.15.92:6835","nonce":383124452}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1871566039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1871566039},{"type":"v1","addr":"172.21.15.92:6837","nonce":1871566039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-19T01:50:49.881618+0000","modified":"2024-03-19T02:03:44.776081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24476,"mds_2":24475,"mds_3":24457,"mds_4":14601},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6839/1427243227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1427243227},{"type":"v1","addr":"172.21.15.92:6839","nonce":1427243227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6841/133114375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6840","nonce":133114375},{"type":"v1","addr":"172.21.15.92:6841","nonce":133114375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3429474924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3429474924},{"type":"v1","addr":"172.21.15.144:6833","nonce":3429474924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/2148025721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2148025721},{"type":"v1","addr":"172.21.15.144:6835","nonce":2148025721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3252974913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3252974913},{"type":"v1","addr":"172.21.15.159:6833","nonce":3252974913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6837/1367568659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1367568659},{"type":"v1","addr":"172.21.15.144:6837","nonce":1367568659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6835/610100257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":610100257},{"type":"v1","addr":"172.21.15.159:6835","nonce":610100257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.159:6837/3903346711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3903346711},{"type":"v1","addr":"172.21.15.159:6837","nonce":3903346711}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6839/380126062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":380126062},{"type":"v1","addr":"172.21.15.144:6839","nonce":380126062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.159:6839/2741545135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2741545135},{"type":"v1","addr":"172.21.15.159:6839","nonce":2741545135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24476,24493,24475,24457,14601]},"id":1}]} 2024-03-19T02:04:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 30 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f28c40a2800 msgr2=0x7f28c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f28c40a2800 0x7f28c40a4cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f28d4000c00 tx=0x7f28d4005fd0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28d81022f0 msgr2=0x7f28d819f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28d81022f0 0x7f28d819f0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f28d000eb10 tx=0x7f28d000c2d0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 shutdown_connections 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28d81022f0 0x7f28d819f0e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d8101930 0x7f28d8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f28c40a2800 0x7f28c40a4cc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 --2- 172.21.15.92:0/3664885929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28d8100740 0x7f28d8197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 >> 172.21.15.92:0/3664885929 conn(0x7f28d80fbcb0 msgr2=0x7f28d80fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 shutdown_connections 2024-03-19T02:04:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:23.965+0000 7f28df997640 1 -- 172.21.15.92:0/3664885929 wait complete. 2024-03-19T02:04:23.976 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:23 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/2561976770' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T02:04:23.976 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:04:23.976 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:23 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3751415025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T02:04:23.976 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:23.976 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:23.976 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:23.976 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:23 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:23.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-19T02:04:24.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:23 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/2561976770' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T02:04:24.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:04:24.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:23 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3751415025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T02:04:24.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:24.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:24.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:24.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:23 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:24.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/2919874065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b41075a0 msgr2=0x7fb7b4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/2919874065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b41075a0 0x7fb7b4109990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009ca0 tx=0x7fb7ac02c0f0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.120 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/2919874065 shutdown_connections 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/2919874065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b41075a0 0x7fb7b4109990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/2919874065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b4104c70 0x7fb7b4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/2919874065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b4102340 0x7fb7b4104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/2919874065 >> 172.21.15.92:0/2919874065 conn(0x7fb7b40fbcb0 msgr2=0x7fb7b40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/2919874065 shutdown_connections 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/2919874065 wait complete. 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.118+0000 7fb7bb0ea640 1 Processor -- start 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7bb0ea640 1 -- start start 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7bb0ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b4102340 0x7fb7b4101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7bb0ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b4104c70 0x7fb7b41002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7bb0ea640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b41075a0 0x7fb7b4100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7bb0ea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b410d4c0 con 0x7fb7b4102340 2024-03-19T02:04:24.121 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7bb0ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b410d340 con 0x7fb7b4104c70 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7bb0ea640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b410d640 con 0x7fb7b41075a0 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b8e86640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b4102340 0x7fb7b4101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b8e86640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b4102340 0x7fb7b4101cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47670/0 (socket says 172.21.15.92:47670) 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b4104c70 0x7fb7b41002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b8e86640 1 -- 172.21.15.92:0/733554318 learned_addr learned my addr 172.21.15.92:0/733554318 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b9687640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b41075a0 0x7fb7b4100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b8e86640 1 -- 172.21.15.92:0/733554318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b41075a0 msgr2=0x7fb7b4100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b8e86640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b41075a0 0x7fb7b4100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b8e86640 1 -- 172.21.15.92:0/733554318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b4104c70 msgr2=0x7fb7b41002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b8e86640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b4104c70 0x7fb7b41002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b8e86640 1 -- 172.21.15.92:0/733554318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b41010c0 con 0x7fb7b4102340 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b3fff640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b4104c70 0x7fb7b41002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b9687640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b41075a0 0x7fb7b4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:04:24.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b8e86640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b4102340 0x7fb7b4101cb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800e9f0 tx=0x7fb7a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.119+0000 7fb7b1ffb640 1 -- 172.21.15.92:0/733554318 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800cd80 con 0x7fb7b4102340 2024-03-19T02:04:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.120+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b41a1c20 con 0x7fb7b4102340 2024-03-19T02:04:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.120+0000 7fb7b1ffb640 1 -- 172.21.15.92:0/733554318 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a8010910 con 0x7fb7b4102340 2024-03-19T02:04:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.120+0000 7fb7b1ffb640 1 -- 172.21.15.92:0/733554318 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8018950 con 0x7fb7b4102340 2024-03-19T02:04:24.123 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.120+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b41a21c0 con 0x7fb7b4102340 2024-03-19T02:04:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.121+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b40fd7a0 con 0x7fb7b4102340 2024-03-19T02:04:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.122+0000 7fb7b1ffb640 1 -- 172.21.15.92:0/733554318 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fb7a8010ac0 con 0x7fb7b4102340 2024-03-19T02:04:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.122+0000 7fb7b1ffb640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7a40a2800 0x7fb7a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.123+0000 7fb7b1ffb640 1 -- 172.21.15.92:0/733554318 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb7a80cf210 con 0x7fb7b4102340 2024-03-19T02:04:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.123+0000 7fb7b3fff640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7a40a2800 0x7fb7a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.124 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.123+0000 7fb7b3fff640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7a40a2800 0x7fb7a40a4cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4101a20 tx=0x7fb7a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.126+0000 7fb7b1ffb640 1 -- 172.21.15.92:0/733554318 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb7a809bc70 con 0x7fb7b4102340 2024-03-19T02:04:24.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:23 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/2561976770' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-19T02:04:24.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-19T02:04:24.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:23 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3751415025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T02:04:24.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:24.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:24.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:24.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:23 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:24.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.351+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7b402d370 con 0x7fb7b4102340 2024-03-19T02:04:24.353 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.353+0000 7fb7b1ffb640 1 -- 172.21.15.92:0/733554318 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22690 (secure 0 0 0) 0x7fb7a809fd10 con 0x7fb7b4102340 2024-03-19T02:04:24.354 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-19T02:04:24.354 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":87,"fsid":"d1e857f4-e591-11ee-95c9-87774f69a715","created":"2024-03-19T01:42:44.963218+0000","modified":"2024-03-19T01:51:40.581546+0000","last_up_change":"2024-03-19T01:49:59.500865+0000","last_in_change":"2024-03-19T01:49:39.131071+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-19T01:46:07.695753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-19T01:50:47.888441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-19T01:50:49.068308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42923908-6c69-4faf-8e1d-36ad4aa578dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6803","nonce":2208105319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6805","nonce":2208105319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6809","nonce":2208105319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2208105319},{"type":"v1","addr":"172.21.15.92:6807","nonce":2208105319}]},"public_addr":"172.21.15.92:6803/2208105319","cluster_addr":"172.21.15.92:6805/2208105319","heartbeat_back_addr":"172.21.15.92:6809/2208105319","heartbeat_front_addr":"172.21.15.92:6807/2208105319","state":["exists","up"]},{"osd":1,"uuid":"2481d494-0d38-4da0-a314-002c20836f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6801","nonce":3690180636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6803","nonce":3690180636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6807","nonce":3690180636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3690180636},{"type":"v1","addr":"172.21.15.144:6805","nonce":3690180636}]},"public_addr":"172.21.15.144:6801/3690180636","cluster_addr":"172.21.15.144:6803/3690180636","heartbeat_back_addr":"172.21.15.144:6807/3690180636","heartbeat_front_addr":"172.21.15.144:6805/3690180636","state":["exists","up"]},{"osd":2,"uuid":"4e6cfae2-9e8c-4588-aa01-0fc24c7297d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6801","nonce":2365994247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6803","nonce":2365994247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6807","nonce":2365994247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2365994247},{"type":"v1","addr":"172.21.15.159:6805","nonce":2365994247}]},"public_addr":"172.21.15.159:6801/2365994247","cluster_addr":"172.21.15.159:6803/2365994247","heartbeat_back_addr":"172.21.15.159:6807/2365994247","heartbeat_front_addr":"172.21.15.159:6805/2365994247","state":["exists","up"]},{"osd":3,"uuid":"eacfff22-7837-4e31-a0da-5d487d903923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6811","nonce":1878896189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6813","nonce":1878896189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6817","nonce":1878896189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":1878896189},{"type":"v1","addr":"172.21.15.92:6815","nonce":1878896189}]},"public_addr":"172.21.15.92:6811/1878896189","cluster_addr":"172.21.15.92:6813/1878896189","heartbeat_back_addr":"172.21.15.92:6817/1878896189","heartbeat_front_addr":"172.21.15.92:6815/1878896189","state":["exists","up"]},{"osd":4,"uuid":"a1f152f6-a121-47d6-8169-0381921ac5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6809","nonce":3335611913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6811","nonce":3335611913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6815","nonce":3335611913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3335611913},{"type":"v1","addr":"172.21.15.144:6813","nonce":3335611913}]},"public_addr":"172.21.15.144:6809/3335611913","cluster_addr":"172.21.15.144:6811/3335611913","heartbeat_back_addr":"172.21.15.144:6815/3335611913","heartbeat_front_addr":"172.21.15.144:6813/3335611913","state":["exists","up"]},{"osd":5,"uuid":"5e86f624-5f0c-4fc4-a12d-1d47750a4428","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6809","nonce":3359154626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6811","nonce":3359154626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6815","nonce":3359154626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3359154626},{"type":"v1","addr":"172.21.15.159:6813","nonce":3359154626}]},"public_addr":"172.21.15.159:6809/3359154626","cluster_addr":"172.21.15.159:6811/3359154626","heartbeat_back_addr":"172.21.15.159:6815/3359154626","heartbeat_front_addr":"172.21.15.159:6813/3359154626","state":["exists","up"]},{"osd":6,"uuid":"e8595632-012c-480c-9324-994266f9a4d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6819","nonce":1165350802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6821","nonce":1165350802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6825","nonce":1165350802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1165350802},{"type":"v1","addr":"172.21.15.92:6823","nonce":1165350802}]},"public_addr":"172.21.15.92:6819/1165350802","cluster_addr":"172.21.15.92:6821/1165350802","heartbeat_back_addr":"172.21.15.92:6825/1165350802","heartbeat_front_addr":"172.21.15.92:6823/1165350802","state":["exists","up"]},{"osd":7,"uuid":"64c441ee-0a42-4022-bb95-ceef4053acac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6817","nonce":2720356070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6819","nonce":2720356070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6823","nonce":2720356070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2720356070},{"type":"v1","addr":"172.21.15.144:6821","nonce":2720356070}]},"public_addr":"172.21.15.144:6817/2720356070","cluster_addr":"172.21.15.144:6819/2720356070","heartbeat_back_addr":"172.21.15.144:6823/2720356070","heartbeat_front_addr":"172.21.15.144:6821/2720356070","state":["exists","up"]},{"osd":8,"uuid":"73e142a5-6d75-48f9-b2f3-6c8f2a7b5a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6817","nonce":3217714611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6819","nonce":3217714611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6823","nonce":3217714611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3217714611},{"type":"v1","addr":"172.21.15.159:6821","nonce":3217714611}]},"public_addr":"172.21.15.159:6817/3217714611","cluster_addr":"172.21.15.159:6819/3217714611","heartbeat_back_addr":"172.21.15.159:6823/3217714611","heartbeat_front_addr":"172.21.15.159:6821/3217714611","state":["exists","up"]},{"osd":9,"uuid":"32cfda0d-94d8-4310-be83-3c0348ef3486","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6827","nonce":4029500126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6829","nonce":4029500126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6833","nonce":4029500126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":4029500126},{"type":"v1","addr":"172.21.15.92:6831","nonce":4029500126}]},"public_addr":"172.21.15.92:6827/4029500126","cluster_addr":"172.21.15.92:6829/4029500126","heartbeat_back_addr":"172.21.15.92:6833/4029500126","heartbeat_front_addr":"172.21.15.92:6831/4029500126","state":["exists","up"]},{"osd":10,"uuid":"8e70e11e-c96c-48bb-916a-4ca295db76e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6825","nonce":3069035738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6827","nonce":3069035738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6831","nonce":3069035738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3069035738},{"type":"v1","addr":"172.21.15.144:6829","nonce":3069035738}]},"public_addr":"172.21.15.144:6825/3069035738","cluster_addr":"172.21.15.144:6827/3069035738","heartbeat_back_addr":"172.21.15.144:6831/3069035738","heartbeat_front_addr":"172.21.15.144:6829/3069035738","state":["exists","up"]},{"osd":11,"uuid":"42fbc0b6-7285-451d-8c11-cb4179823811","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6825","nonce":3616019688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6827","nonce":3616019688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6831","nonce":3616019688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3616019688},{"type":"v1","addr":"172.21.15.159:6829","nonce":3616019688}]},"public_addr":"172.21.15.159:6825/3616019688","cluster_addr":"172.21.15.159:6827/3616019688","heartbeat_back_addr":"172.21.15.159:6831/3616019688","heartbeat_front_addr":"172.21.15.159:6829/3616019688","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:16.923668+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:45:40.859116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:04.512347+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:29.831582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:46:55.212893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:20.818524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:47:46.545416+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:12.637677+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:48:38.415265+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:04.823308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:31.117765+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-19T01:49:57.842720+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:0/1888848283":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1866676846":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6800/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/239005224":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/2465427523":"2024-03-20T01:43:39.363114+0000","172.21.15.92:6801/1465441882":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3016291750":"2024-03-20T01:43:12.221209+0000","172.21.15.92:0/3316636845":"2024-03-20T01:43:39.363114+0000","172.21.15.92:0/3619099089":"2024-03-20T01:43:12.221209+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-19T02:04:24.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.355+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7a40a2800 msgr2=0x7fb7a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.355+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7a40a2800 0x7fb7a40a4cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4101a20 tx=0x7fb7a0005f90 comp rx=0 tx=0).stop 2024-03-19T02:04:24.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.355+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b4102340 msgr2=0x7fb7b4101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.356+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b4102340 0x7fb7b4101cb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800e9f0 tx=0x7fb7a800eeb0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.356+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 shutdown_connections 2024-03-19T02:04:24.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.356+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fb7a40a2800 0x7fb7a40a4cc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.356 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.356+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b41075a0 0x7fb7b4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.356+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b4104c70 0x7fb7b41002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.356+0000 7fb7bb0ea640 1 --2- 172.21.15.92:0/733554318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7b4102340 0x7fb7b4101cb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.356+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 >> 172.21.15.92:0/733554318 conn(0x7fb7b40fbcb0 msgr2=0x7fb7b4102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:24.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.356+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 shutdown_connections 2024-03-19T02:04:24.357 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.356+0000 7fb7bb0ea640 1 -- 172.21.15.92:0/733554318 wait complete. 2024-03-19T02:04:24.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-19T02:04:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.481+0000 7f251b95b640 1 -- 172.21.15.92:0/616621447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25141022d0 msgr2=0x7f2514108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.481+0000 7f251b95b640 1 --2- 172.21.15.92:0/616621447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25141022d0 0x7f2514108b60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2510009900 tx=0x7f251002bc30 comp rx=0 tx=0).stop 2024-03-19T02:04:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.481+0000 7f251b95b640 1 -- 172.21.15.92:0/616621447 shutdown_connections 2024-03-19T02:04:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.481+0000 7f251b95b640 1 --2- 172.21.15.92:0/616621447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25141022d0 0x7f2514108b60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.484 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.481+0000 7f251b95b640 1 --2- 172.21.15.92:0/616621447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2514101910 0x7f2514101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 --2- 172.21.15.92:0/616621447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2514100720 0x7f2514100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 -- 172.21.15.92:0/616621447 >> 172.21.15.92:0/616621447 conn(0x7f25140fbcb0 msgr2=0x7f25140fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 -- 172.21.15.92:0/616621447 shutdown_connections 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 -- 172.21.15.92:0/616621447 wait complete. 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 Processor -- start 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 -- start start 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514100720 0x7f2514069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2514101910 0x7f251419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25141022d0 0x7f251419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f251410c380 con 0x7f2514100720 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f251410c200 con 0x7f25141022d0 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f251b95b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f251410c500 con 0x7f2514101910 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f25196f7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514100720 0x7f2514069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f25196f7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514100720 0x7f2514069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47694/0 (socket says 172.21.15.92:47694) 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f25196f7640 1 -- 172.21.15.92:0/1190410532 learned_addr learned my addr 172.21.15.92:0/1190410532 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.482+0000 7f2518ef6640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2514101910 0x7f251419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f2519ef8640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25141022d0 0x7f251419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f25196f7640 1 -- 172.21.15.92:0/1190410532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2514101910 msgr2=0x7f251419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f25196f7640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2514101910 0x7f251419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f25196f7640 1 -- 172.21.15.92:0/1190410532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25141022d0 msgr2=0x7f251419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f25196f7640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25141022d0 0x7f251419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f25196f7640 1 -- 172.21.15.92:0/1190410532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251419f750 con 0x7f2514100720 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f2519ef8640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25141022d0 0x7f251419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f25196f7640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514100720 0x7f2514069510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f250c00ebc0 tx=0x7f250c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f25067fc640 1 -- 172.21.15.92:0/1190410532 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c003b70 con 0x7f2514100720 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f25067fc640 1 -- 172.21.15.92:0/1190410532 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c003d10 con 0x7f2514100720 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f25067fc640 1 -- 172.21.15.92:0/1190410532 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c019980 con 0x7f2514100720 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251419fa40 con 0x7f2514100720 2024-03-19T02:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.483+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251419ff90 con 0x7f2514100720 2024-03-19T02:04:24.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.484+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2514107a40 con 0x7f2514100720 2024-03-19T02:04:24.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.485+0000 7f25067fc640 1 -- 172.21.15.92:0/1190410532 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f250c019b20 con 0x7f2514100720 2024-03-19T02:04:24.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.486+0000 7f25067fc640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25000a2800 0x7f25000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.487 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.486+0000 7f2518ef6640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25000a2800 0x7f25000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.486+0000 7f25067fc640 1 -- 172.21.15.92:0/1190410532 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f250c0cf480 con 0x7f2514100720 2024-03-19T02:04:24.488 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.486+0000 7f2518ef6640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25000a2800 0x7f25000a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2514069280 tx=0x7f2508005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:24.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.489+0000 7f25067fc640 1 -- 172.21.15.92:0/1190410532 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f250c09c560 con 0x7f2514100720 2024-03-19T02:04:24.711 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.711+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f251402d290 con 0x7f2514100720 2024-03-19T02:04:24.713 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.713+0000 7f25067fc640 1 -- 172.21.15.92:0/1190410532 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f250c09be60 con 0x7f2514100720 2024-03-19T02:04:24.713 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-19T02:04:24.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25000a2800 msgr2=0x7f25000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25000a2800 0x7f25000a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2514069280 tx=0x7f2508005f90 comp rx=0 tx=0).stop 2024-03-19T02:04:24.717 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514100720 msgr2=0x7f2514069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514100720 0x7f2514069510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f250c00ebc0 tx=0x7f250c00c6a0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 shutdown_connections 2024-03-19T02:04:24.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25141022d0 0x7f251419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f25000a2800 0x7f25000a4cc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2514101910 0x7f251419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 --2- 172.21.15.92:0/1190410532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2514100720 0x7f2514069510 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 >> 172.21.15.92:0/1190410532 conn(0x7f25140fbcb0 msgr2=0x7f2514104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:24.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 shutdown_connections 2024-03-19T02:04:24.718 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.717+0000 7f251b95b640 1 -- 172.21.15.92:0/1190410532 wait complete. 2024-03-19T02:04:24.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-19T02:04:24.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.840+0000 7f2ce76bf640 1 -- 172.21.15.92:0/2645460765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 msgr2=0x7f2ce0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.840+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/2645460765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 0x7f2ce0100b40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009ca0 tx=0x7f2cd802c0f0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.840+0000 7f2ce76bf640 1 -- 172.21.15.92:0/2645460765 shutdown_connections 2024-03-19T02:04:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.840+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/2645460765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce01022d0 0x7f2ce0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.840+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/2645460765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0101910 0x7f2ce0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.840+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/2645460765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 0x7f2ce0100b40 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.840+0000 7f2ce76bf640 1 -- 172.21.15.92:0/2645460765 >> 172.21.15.92:0/2645460765 conn(0x7f2ce00fbcb0 msgr2=0x7f2ce00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.840+0000 7f2ce76bf640 1 -- 172.21.15.92:0/2645460765 shutdown_connections 2024-03-19T02:04:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.840+0000 7f2ce76bf640 1 -- 172.21.15.92:0/2645460765 wait complete. 2024-03-19T02:04:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce76bf640 1 Processor -- start 2024-03-19T02:04:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce76bf640 1 -- start start 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce76bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 0x7f2ce0069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce76bf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0101910 0x7f2ce019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce76bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce01022d0 0x7f2ce019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce76bf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce010c390 con 0x7f2ce0100720 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce76bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce010c210 con 0x7f2ce01022d0 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce76bf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce010c510 con 0x7f2ce0101910 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce545b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 0x7f2ce0069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce545b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 0x7f2ce0069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47744/0 (socket says 172.21.15.92:47744) 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce545b640 1 -- 172.21.15.92:0/388551597 learned_addr learned my addr 172.21.15.92:0/388551597 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce4c5a640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0101910 0x7f2ce019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce545b640 1 -- 172.21.15.92:0/388551597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0101910 msgr2=0x7f2ce019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce545b640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0101910 0x7f2ce019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.842+0000 7f2ce545b640 1 -- 172.21.15.92:0/388551597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce01022d0 msgr2=0x7f2ce019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:24.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.841+0000 7f2ce5c5c640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce01022d0 0x7f2ce019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.842+0000 7f2ce545b640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce01022d0 0x7f2ce019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:24.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.842+0000 7f2ce545b640 1 -- 172.21.15.92:0/388551597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce019f7c0 con 0x7f2ce0100720 2024-03-19T02:04:24.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.842+0000 7f2ce545b640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 0x7f2ce0069510 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80324c0 tx=0x7f2cd8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:24.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.842+0000 7f2cd27fc640 1 -- 172.21.15.92:0/388551597 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8039040 con 0x7f2ce0100720 2024-03-19T02:04:24.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.842+0000 7f2cd27fc640 1 -- 172.21.15.92:0/388551597 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd8044070 con 0x7f2ce0100720 2024-03-19T02:04:24.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.842+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce019fa50 con 0x7f2ce0100720 2024-03-19T02:04:24.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.842+0000 7f2cd27fc640 1 -- 172.21.15.92:0/388551597 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80042a0 con 0x7f2ce0100720 2024-03-19T02:04:24.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.842+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce019ff40 con 0x7f2ce0100720 2024-03-19T02:04:24.846 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.843+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce0107b10 con 0x7f2ce0100720 2024-03-19T02:04:24.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.844+0000 7f2cd27fc640 1 -- 172.21.15.92:0/388551597 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f2cd8003970 con 0x7f2ce0100720 2024-03-19T02:04:24.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.844+0000 7f2cd27fc640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ccc0a2800 0x7f2ccc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:24.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.844+0000 7f2ce4c5a640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ccc0a2800 0x7f2ccc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:24.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.844+0000 7f2cd27fc640 1 -- 172.21.15.92:0/388551597 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8641+0+0 (secure 0 0 0) 0x7f2cd80f1b20 con 0x7f2ce0100720 2024-03-19T02:04:24.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.845+0000 7f2ce4c5a640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ccc0a2800 0x7f2ccc0a4cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0069280 tx=0x7f2cd400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:24.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:24.848+0000 7f2cd27fc640 1 -- 172.21.15.92:0/388551597 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2cd80beab0 con 0x7f2ce0100720 2024-03-19T02:04:24.998 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h[97570]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:24.998 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h[97570]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:24.998 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mds[97574]: -- 172.21.15.144:0/3592848368 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56397886da40 con 0x563978840c00 2024-03-19T02:04:24.998 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h[97570]: starting mds.h at 2024-03-19T02:04:24.998 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e[97263]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:24.998 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e[97263]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:24.999 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mds[97267]: -- 172.21.15.144:0/3793141513 <== mon.2 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5572e1e644e0 con 0x5572e1e3ac00 2024-03-19T02:04:24.999 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mds[97267]: -- 172.21.15.144:0/3793141513 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572e1e67a40 con 0x5572e1e3ac00 2024-03-19T02:04:24.999 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e[97263]: starting mds.e at 2024-03-19T02:04:24.999 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k[96942]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:24.999 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k[96942]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:24.999 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mds[96946]: -- 172.21.15.144:0/1064707752 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559abf6e1a40 con 0x559abf6b4800 2024-03-19T02:04:24.999 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k[96942]: starting mds.k at 2024-03-19T02:04:25.000 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b[97876]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:25.000 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b[97876]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:25.000 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mds[97880]: -- 172.21.15.144:0/2508048228 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ec77cc4e0 con 0x562ec77a3000 2024-03-19T02:04:25.000 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mds[97880]: -- 172.21.15.144:0/2508048228 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ec77cfa40 con 0x562ec77a3000 2024-03-19T02:04:25.000 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b[97876]: starting mds.b at 2024-03-19T02:04:25.000 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: pgmap v699: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-19T02:04:25.000 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/3664885929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T02:04:25.000 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/733554318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T02:04:25.000 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:04:25.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:04:25.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:25.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1190410532' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T02:04:25.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-19T02:04:25.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T02:04:25.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/1190410532' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-19T02:04:25.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: osdmap e88: 12 total, 12 up, 12 in 2024-03-19T02:04:25.001 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:24 smithi144 ceph-mon[67199]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-19T02:04:25.060 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a[104547]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:25.060 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a[104547]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:25.060 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mds[104551]: -- 172.21.15.92:0/1691110293 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55596f1cba40 con 0x55596f19e800 2024-03-19T02:04:25.060 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a[104547]: starting mds.a at 2024-03-19T02:04:25.061 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g[104187]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:25.061 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g[104187]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:25.061 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mds[104191]: -- 172.21.15.92:0/2065738874 <== mon.0 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d24cffba40 con 0x55d24cfce800 2024-03-19T02:04:25.061 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g[104187]: starting mds.g at 2024-03-19T02:04:25.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a[62570]: 2024-03-19T02:04:24.712+0000 7f55cd605700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T02:04:25.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: pgmap v699: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/3664885929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/733554318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1190410532' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/1190410532' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: osdmap e88: 12 total, 12 up, 12 in 2024-03-19T02:04:25.062 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:24 smithi092 ceph-mon[62574]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-19T02:04:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.077+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2ce002d440 con 0x7f2ce0100720 2024-03-19T02:04:25.205 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c[98418]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:25.205 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c[98418]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:25.206 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mds[98422]: -- 172.21.15.159:0/3681439226 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac97945a40 con 0x55ac97918800 2024-03-19T02:04:25.206 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c[98418]: starting mds.c at 2024-03-19T02:04:25.206 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l[97804]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:25.206 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l[97804]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:25.206 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mds[97808]: -- 172.21.15.159:0/2986035155 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d3b0f9a40 con 0x562d3b0cc800 2024-03-19T02:04:25.206 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l[97804]: starting mds.l at 2024-03-19T02:04:25.206 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f[98119]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:25.206 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f[98119]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:25.207 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mds[98123]: -- 172.21.15.159:0/343378803 <== mon.2 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8dc425a40 con 0x55a8dc3f9000 2024-03-19T02:04:25.207 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f[98119]: starting mds.f at 2024-03-19T02:04:25.207 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: ignoring --setuser ceph since I am not root 2024-03-19T02:04:25.207 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: ignoring --setgroup ceph since I am not root 2024-03-19T02:04:25.207 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mds[98730]: -- 172.21.15.159:0/2428309550 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588df3d7a40 con 0x5588df3aa800 2024-03-19T02:04:25.207 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: starting mds.i at 2024-03-19T02:04:25.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: pgmap v699: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-19T02:04:25.207 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/3664885929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/733554318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1190410532' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/1190410532' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: osdmap e88: 12 total, 12 up, 12 in 2024-03-19T02:04:25.208 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:24 smithi159 ceph-mon[67110]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-19T02:04:25.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.749+0000 7f2cd27fc640 1 -- 172.21.15.92:0/388551597 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f2cd80be3b0 con 0x7f2ce0100720 2024-03-19T02:04:25.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ccc0a2800 msgr2=0x7f2ccc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:25.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ccc0a2800 0x7f2ccc0a4cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0069280 tx=0x7f2cd400b390 comp rx=0 tx=0).stop 2024-03-19T02:04:25.753 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 msgr2=0x7f2ce0069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:25.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 0x7f2ce0069510 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80324c0 tx=0x7f2cd8032f20 comp rx=0 tx=0).stop 2024-03-19T02:04:25.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 shutdown_connections 2024-03-19T02:04:25.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce01022d0 0x7f2ce019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:25.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f2ccc0a2800 0x7f2ccc0a4cc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:25.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0101910 0x7f2ce019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:25.754 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 --2- 172.21.15.92:0/388551597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ce0100720 0x7f2ce0069510 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:25.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 >> 172.21.15.92:0/388551597 conn(0x7f2ce00fbcb0 msgr2=0x7f2ce0104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:25.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 shutdown_connections 2024-03-19T02:04:25.755 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.752+0000 7f2ce76bf640 1 -- 172.21.15.92:0/388551597 wait complete. 2024-03-19T02:04:25.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-19T02:04:25.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.879+0000 7fce23190640 1 -- 172.21.15.92:0/926038093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c101910 msgr2=0x7fce1c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:25.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.879+0000 7fce23190640 1 --2- 172.21.15.92:0/926038093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c101910 0x7fce1c101d90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fce08009900 tx=0x7fce0802bc30 comp rx=0 tx=0).stop 2024-03-19T02:04:25.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.879+0000 7fce23190640 1 -- 172.21.15.92:0/926038093 shutdown_connections 2024-03-19T02:04:25.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.879+0000 7fce23190640 1 --2- 172.21.15.92:0/926038093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce1c1022d0 0x7fce1c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:25.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.879+0000 7fce23190640 1 --2- 172.21.15.92:0/926038093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c101910 0x7fce1c101d90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:25.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.879+0000 7fce23190640 1 --2- 172.21.15.92:0/926038093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce1c100720 0x7fce1c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:25.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.879+0000 7fce23190640 1 -- 172.21.15.92:0/926038093 >> 172.21.15.92:0/926038093 conn(0x7fce1c0fbcb0 msgr2=0x7fce1c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:25.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.879+0000 7fce23190640 1 -- 172.21.15.92:0/926038093 shutdown_connections 2024-03-19T02:04:25.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.879+0000 7fce23190640 1 -- 172.21.15.92:0/926038093 wait complete. 2024-03-19T02:04:25.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce23190640 1 Processor -- start 2024-03-19T02:04:25.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce23190640 1 -- start start 2024-03-19T02:04:25.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce23190640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce1c100720 0x7fce1c1046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:25.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce23190640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce1c101910 0x7fce1c102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:25.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce23190640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c1022d0 0x7fce1c103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:25.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce23190640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce1c10c410 con 0x7fce1c1022d0 2024-03-19T02:04:25.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce23190640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fce1c10c290 con 0x7fce1c100720 2024-03-19T02:04:25.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce23190640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce1c10c590 con 0x7fce1c101910 2024-03-19T02:04:25.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce2172d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c1022d0 0x7fce1c103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:25.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce2172d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c1022d0 0x7fce1c103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47768/0 (socket says 172.21.15.92:47768) 2024-03-19T02:04:25.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce2172d640 1 -- 172.21.15.92:0/246807087 learned_addr learned my addr 172.21.15.92:0/246807087 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:25.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce20f2c640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce1c100720 0x7fce1c1046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:25.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce1bfff640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce1c101910 0x7fce1c102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:25.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce2172d640 1 -- 172.21.15.92:0/246807087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce1c101910 msgr2=0x7fce1c102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:25.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce2172d640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce1c101910 0x7fce1c102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:25.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce2172d640 1 -- 172.21.15.92:0/246807087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce1c100720 msgr2=0x7fce1c1046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:25.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce2172d640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce1c100720 0x7fce1c1046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:25.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce2172d640 1 -- 172.21.15.92:0/246807087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1c103ac0 con 0x7fce1c1022d0 2024-03-19T02:04:25.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.880+0000 7fce2172d640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c1022d0 0x7fce1c103230 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fce1400eaa0 tx=0x7fce1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:25.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.881+0000 7fce19ffb640 1 -- 172.21.15.92:0/246807087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1400cdb0 con 0x7fce1c1022d0 2024-03-19T02:04:25.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.881+0000 7fce19ffb640 1 -- 172.21.15.92:0/246807087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce14005020 con 0x7fce1c1022d0 2024-03-19T02:04:25.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.881+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1c1a1d10 con 0x7fce1c1022d0 2024-03-19T02:04:25.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.881+0000 7fce19ffb640 1 -- 172.21.15.92:0/246807087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce140116b0 con 0x7fce1c1022d0 2024-03-19T02:04:25.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.881+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce1c1a21b0 con 0x7fce1c1022d0 2024-03-19T02:04:25.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.882+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce1c1a23e0 con 0x7fce1c1022d0 2024-03-19T02:04:25.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.883+0000 7fce19ffb640 1 -- 172.21.15.92:0/246807087 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fce14005560 con 0x7fce1c1022d0 2024-03-19T02:04:25.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.883+0000 7fce19ffb640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fce0c0a2800 0x7fce0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:25.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.883+0000 7fce20f2c640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fce0c0a2800 0x7fce0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:25.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.883+0000 7fce19ffb640 1 -- 172.21.15.92:0/246807087 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8641+0+0 (secure 0 0 0) 0x7fce140cef60 con 0x7fce1c1022d0 2024-03-19T02:04:25.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.884+0000 7fce20f2c640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fce0c0a2800 0x7fce0c0a4cc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fce1c101770 tx=0x7fce10008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:25.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:25.888+0000 7fce19ffb640 1 -- 172.21.15.92:0/246807087 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fce14016030 con 0x7fce1c1022d0 2024-03-19T02:04:26.015 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:25 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/388551597' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T02:04:26.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:25 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/388551597' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T02:04:26.106 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:25 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/388551597' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-19T02:04:26.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.115+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fce1c1a25b0 con 0x7fce1c1022d0 2024-03-19T02:04:26.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.756+0000 7fce19ffb640 1 -- 172.21.15.92:0/246807087 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7fce1409be60 con 0x7fce1c1022d0 2024-03-19T02:04:26.757 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2024-03-19T02:04:26.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.759+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fce0c0a2800 msgr2=0x7fce0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:26.760 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.759+0000 7fce23190640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fce0c0a2800 0x7fce0c0a4cc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fce1c101770 tx=0x7fce10008040 comp rx=0 tx=0).stop 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.759+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c1022d0 msgr2=0x7fce1c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.759+0000 7fce23190640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c1022d0 0x7fce1c103230 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fce1400eaa0 tx=0x7fce1400ef60 comp rx=0 tx=0).stop 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.760+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 shutdown_connections 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.760+0000 7fce23190640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce1c1022d0 0x7fce1c103230 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.760+0000 7fce23190640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7fce0c0a2800 0x7fce0c0a4cc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.760+0000 7fce23190640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce1c101910 0x7fce1c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.760+0000 7fce23190640 1 --2- 172.21.15.92:0/246807087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce1c100720 0x7fce1c1046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.760+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 >> 172.21.15.92:0/246807087 conn(0x7fce1c0fbcb0 msgr2=0x7fce1c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.760+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 shutdown_connections 2024-03-19T02:04:26.761 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.760+0000 7fce23190640 1 -- 172.21.15.92:0/246807087 wait complete. 2024-03-19T02:04:26.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-19T02:04:26.882 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 -- 172.21.15.92:0/1734960340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974102ad0 msgr2=0x7f7974102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 --2- 172.21.15.92:0/1734960340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974102ad0 0x7f7974102f50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7968009930 tx=0x7f796802bc30 comp rx=0 tx=0).stop 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 -- 172.21.15.92:0/1734960340 shutdown_connections 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 --2- 172.21.15.92:0/1734960340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7974103490 0x7f7974109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 --2- 172.21.15.92:0/1734960340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974102ad0 0x7f7974102f50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 --2- 172.21.15.92:0/1734960340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79740ffb00 0x7f79740fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 -- 172.21.15.92:0/1734960340 >> 172.21.15.92:0/1734960340 conn(0x7f79740fb410 msgr2=0x7f79740fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 -- 172.21.15.92:0/1734960340 shutdown_connections 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 -- 172.21.15.92:0/1734960340 wait complete. 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.880+0000 7f79793ab640 1 Processor -- start 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79793ab640 1 -- start start 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79793ab640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79740ffb00 0x7f7974102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79793ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7974102ad0 0x7f7974100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79793ab640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974103490 0x7f7974100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79793ab640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f797410d4c0 con 0x7f7974103490 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79793ab640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f797410d340 con 0x7f7974102ad0 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79793ab640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f797410d640 con 0x7f79740ffb00 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974103490 0x7f7974100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79737fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974103490 0x7f7974100fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47784/0 (socket says 172.21.15.92:47784) 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79727fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7974102ad0 0x7f7974100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79737fe640 1 -- 172.21.15.92:0/836758105 learned_addr learned my addr 172.21.15.92:0/836758105 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f7972ffd640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79740ffb00 0x7f7974102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79737fe640 1 -- 172.21.15.92:0/836758105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79740ffb00 msgr2=0x7f7974102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79737fe640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79740ffb00 0x7f7974102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79737fe640 1 -- 172.21.15.92:0/836758105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7974102ad0 msgr2=0x7f7974100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79737fe640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7974102ad0 0x7f7974100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79737fe640 1 -- 172.21.15.92:0/836758105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7974101850 con 0x7f7974103490 2024-03-19T02:04:26.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79727fc640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7974102ad0 0x7f7974100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-19T02:04:26.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79737fe640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974103490 0x7f7974100fc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f796c00eb10 tx=0x7f796c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:26.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f795bfff640 1 -- 172.21.15.92:0/836758105 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c004b20 con 0x7f7974103490 2024-03-19T02:04:26.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79741a1c90 con 0x7f7974103490 2024-03-19T02:04:26.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.881+0000 7f795bfff640 1 -- 172.21.15.92:0/836758105 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f796c004cc0 con 0x7f7974103490 2024-03-19T02:04:26.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.882+0000 7f795bfff640 1 -- 172.21.15.92:0/836758105 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c010760 con 0x7f7974103490 2024-03-19T02:04:26.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.882+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79741a21b0 con 0x7f7974103490 2024-03-19T02:04:26.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.882+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7974068a60 con 0x7f7974103490 2024-03-19T02:04:26.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.884+0000 7f795bfff640 1 -- 172.21.15.92:0/836758105 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f796c010900 con 0x7f7974103490 2024-03-19T02:04:26.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.884+0000 7f795bfff640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f795c0a2800 0x7f795c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-19T02:04:26.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.884+0000 7f7972ffd640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f795c0a2800 0x7f795c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-19T02:04:26.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.884+0000 7f795bfff640 1 -- 172.21.15.92:0/836758105 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8145+0+0 (secure 0 0 0) 0x7f796c0cf230 con 0x7f7974103490 2024-03-19T02:04:26.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.885+0000 7f7972ffd640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f795c0a2800 0x7f795c0a4cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7964005fd0 tx=0x7f7964005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-19T02:04:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:26.892+0000 7f795bfff640 1 -- 172.21.15.92:0/836758105 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f796c053740 con 0x7f7974103490 2024-03-19T02:04:27.033 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: pgmap v701: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: Cluster is now healthy 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/388551597' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.92:6838/2561430846,v1:172.21.15.92:6841/2561430846] up:boot 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.92:6839/1078364778,v1:172.21.15.92:6840/1078364778] up:boot 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.144:6833/143825311,v1:172.21.15.144:6835/143825311] up:boot 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6836/363104946,v1:172.21.15.159:6838/363104946] up:boot 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.144:6836/2913466628,v1:172.21.15.144:6837/2913466628] up:boot 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.144:6838/4001528299,v1:172.21.15.144:6839/4001528299] up:boot 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6832/3263545117,v1:172.21.15.159:6834/3263545117] up:boot 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6833/766758381,v1:172.21.15.159:6835/766758381] up:boot 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.144:6832/807885213,v1:172.21.15.144:6834/807885213] up:boot 2024-03-19T02:04:27.034 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: mds.? [v2:172.21.15.159:6837/1396315742,v1:172.21.15.159:6839/1396315742] up:boot 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: fsmap 12 up:standby 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T02:04:27.035 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:26 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/246807087' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T02:04:27.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.073+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7974108c80 con 0x7f7974103490 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: pgmap v701: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: Cluster is now healthy 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/388551597' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.92:6838/2561430846,v1:172.21.15.92:6841/2561430846] up:boot 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.92:6839/1078364778,v1:172.21.15.92:6840/1078364778] up:boot 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.144:6833/143825311,v1:172.21.15.144:6835/143825311] up:boot 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6836/363104946,v1:172.21.15.159:6838/363104946] up:boot 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.144:6836/2913466628,v1:172.21.15.144:6837/2913466628] up:boot 2024-03-19T02:04:27.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.144:6838/4001528299,v1:172.21.15.144:6839/4001528299] up:boot 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6832/3263545117,v1:172.21.15.159:6834/3263545117] up:boot 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6833/766758381,v1:172.21.15.159:6835/766758381] up:boot 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.144:6832/807885213,v1:172.21.15.144:6834/807885213] up:boot 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: mds.? [v2:172.21.15.159:6837/1396315742,v1:172.21.15.159:6839/1396315742] up:boot 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: fsmap 12 up:standby 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T02:04:27.098 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T02:04:27.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T02:04:27.099 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:26 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/246807087' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T02:04:27.124 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: pgmap v701: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-19T02:04:27.124 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-19T02:04:27.124 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: Cluster is now healthy 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/388551597' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.92:6838/2561430846,v1:172.21.15.92:6841/2561430846] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.92:6839/1078364778,v1:172.21.15.92:6840/1078364778] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.144:6833/143825311,v1:172.21.15.144:6835/143825311] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6836/363104946,v1:172.21.15.159:6838/363104946] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.144:6836/2913466628,v1:172.21.15.144:6837/2913466628] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.144:6838/4001528299,v1:172.21.15.144:6839/4001528299] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6832/3263545117,v1:172.21.15.159:6834/3263545117] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6833/766758381,v1:172.21.15.159:6835/766758381] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.144:6832/807885213,v1:172.21.15.144:6834/807885213] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: mds.? [v2:172.21.15.159:6837/1396315742,v1:172.21.15.159:6839/1396315742] up:boot 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: fsmap 12 up:standby 2024-03-19T02:04:27.125 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='mgr.14152 172.21.15.92:0/2348117528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-19T02:04:27.126 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:26 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/246807087' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T02:04:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.761+0000 7f795bfff640 1 -- 172.21.15.92:0/836758105 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7f796c09c370 con 0x7f7974103490 2024-03-19T02:04:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2024-03-19T02:04:27.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f795c0a2800 msgr2=0x7f795c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:27.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f795c0a2800 0x7f795c0a4cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7964005fd0 tx=0x7f7964005850 comp rx=0 tx=0).stop 2024-03-19T02:04:27.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974103490 msgr2=0x7f7974100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-19T02:04:27.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974103490 0x7f7974100fc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f796c00eb10 tx=0x7f796c00c2d0 comp rx=0 tx=0).stop 2024-03-19T02:04:27.765 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 shutdown_connections 2024-03-19T02:04:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7974103490 0x7f7974100fc0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7974102ad0 0x7f7974100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.92:6800/3230736370,v1:172.21.15.92:6801/3230736370] conn(0x7f795c0a2800 0x7f795c0a4cc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 --2- 172.21.15.92:0/836758105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79740ffb00 0x7f7974102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-19T02:04:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 >> 172.21.15.92:0/836758105 conn(0x7f79740fb410 msgr2=0x7f7974105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-19T02:04:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 shutdown_connections 2024-03-19T02:04:27.766 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-19T02:04:27.765+0000 7f79793ab640 1 -- 172.21.15.92:0/836758105 wait complete. 2024-03-19T02:04:27.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T02:04:27.806 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T02:04:27.839 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T02:04:27.868 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-19T02:04:27.869 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-19T02:04:27.883 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-19T02:04:27.899 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-19T02:04:27.924 INFO:tasks.cephadm:Stopping all daemons... 2024-03-19T02:04:27.924 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-19T02:04:27.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.a 2024-03-19T02:04:28.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:27 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/246807087' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T02:04:28.060 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:27 smithi092 ceph-mon[62574]: osdmap e89: 12 total, 12 up, 12 in 2024-03-19T02:04:28.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:27 smithi092 ceph-mon[62574]: from='client.? 172.21.15.92:0/836758105' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T02:04:28.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:27 smithi092 ceph-mon[62574]: pgmap v703: 65 pgs: 65 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T02:04:28.061 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:27 smithi092 systemd[1]: Stopping Ceph mon.a for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:04:28.096 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:27 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/246807087' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T02:04:28.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:27 smithi144 ceph-mon[67199]: osdmap e89: 12 total, 12 up, 12 in 2024-03-19T02:04:28.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:27 smithi144 ceph-mon[67199]: from='client.? 172.21.15.92:0/836758105' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T02:04:28.097 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:27 smithi144 ceph-mon[67199]: pgmap v703: 65 pgs: 65 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T02:04:28.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:27 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/246807087' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-19T02:04:28.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:27 smithi159 ceph-mon[67110]: osdmap e89: 12 total, 12 up, 12 in 2024-03-19T02:04:28.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:27 smithi159 ceph-mon[67110]: from='client.? 172.21.15.92:0/836758105' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-19T02:04:28.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:27 smithi159 ceph-mon[67110]: pgmap v703: 65 pgs: 65 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-19T02:04:28.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a[62570]: 2024-03-19T02:04:28.237+0000 7f55d2e10700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:04:28.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a[62570]: 2024-03-19T02:04:28.237+0000 7f55d2e10700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-19T02:04:28.560 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:28 smithi092 podman[121623]: 2024-03-19 02:04:28.319103587 +0000 UTC m=+0.209658716 container died 9b3aea2ff4cd4cee522905e2fa6c6b332d63335d29da091fe46ffaa83344b750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True) 2024-03-19T02:04:28.947 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:28 smithi092 podman[121623]: 2024-03-19 02:04:28.713135467 +0000 UTC m=+0.603690607 container cleanup 9b3aea2ff4cd4cee522905e2fa6c6b332d63335d29da091fe46ffaa83344b750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-19T02:04:28.948 INFO:journalctl@ceph.mon.a.smithi092.stdout:Mar 19 02:04:28 smithi092 bash[121623]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-a 2024-03-19T02:04:29.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.a.service' 2024-03-19T02:04:29.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:04:29.120 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-19T02:04:29.120 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-19T02:04:29.120 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.b 2024-03-19T02:04:29.440 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:29 smithi144 systemd[1]: Stopping Ceph mon.b for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:04:29.846 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:29 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-b[67194]: 2024-03-19T02:04:29.439+0000 7f06fe07d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:04:29.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:29 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-b[67194]: 2024-03-19T02:04:29.439+0000 7f06fe07d700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-19T02:04:29.847 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:29 smithi144 podman[103152]: 2024-03-19 02:04:29.542731838 +0000 UTC m=+0.227220230 container died 63cf629954ae20d7ab0db9a02d9635ea9014f307daae80b74f8c4599fbf88e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-03-19T02:04:30.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:29 smithi144 podman[103152]: 2024-03-19 02:04:29.974572758 +0000 UTC m=+0.659061141 container cleanup 63cf629954ae20d7ab0db9a02d9635ea9014f307daae80b74f8c4599fbf88e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-19T02:04:30.201 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:29 smithi144 bash[103152]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-b 2024-03-19T02:04:30.442 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.b.service' 2024-03-19T02:04:30.467 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:30 smithi144 podman[103166]: 2024-03-19 02:04:30.201439229 +0000 UTC m=+0.657735243 container remove 63cf629954ae20d7ab0db9a02d9635ea9014f307daae80b74f8c4599fbf88e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-b, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-19T02:04:30.467 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:30 smithi144 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-19T02:04:30.467 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:30 smithi144 systemd[1]: Stopped Ceph mon.b for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T02:04:30.467 INFO:journalctl@ceph.mon.b.smithi144.stdout:Mar 19 02:04:30 smithi144 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.b.service: Consumed 10.652s CPU time. 2024-03-19T02:04:30.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:04:30.498 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-19T02:04:30.499 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-19T02:04:30.499 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.c 2024-03-19T02:04:30.811 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:30 smithi159 systemd[1]: Stopping Ceph mon.c for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:04:31.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:30 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-c[67106]: 2024-03-19T02:04:30.811+0000 7fc125b3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:04:31.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:30 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-c[67106]: 2024-03-19T02:04:30.811+0000 7fc125b3e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-19T02:04:31.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:30 smithi159 podman[103137]: 2024-03-19 02:04:30.954588573 +0000 UTC m=+0.260855307 container died 31e125f9f5c4c78d9bae217fe93b48b9a426a5f2b18374da8176f39ece1a8074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-c, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:04:31.605 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:31 smithi159 podman[103137]: 2024-03-19 02:04:31.33735266 +0000 UTC m=+0.643619402 container cleanup 31e125f9f5c4c78d9bae217fe93b48b9a426a5f2b18374da8176f39ece1a8074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-19T02:04:31.605 INFO:journalctl@ceph.mon.c.smithi159.stdout:Mar 19 02:04:31 smithi159 bash[103137]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mon-c 2024-03-19T02:04:31.758 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mon.c.service' 2024-03-19T02:04:31.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:04:31.811 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-19T02:04:31.811 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-19T02:04:31.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.x 2024-03-19T02:04:32.143 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 02:04:31 smithi092 systemd[1]: Stopping Ceph mgr.x for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:04:32.544 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 02:04:32 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T02:04:32.142+0000 7fcf91e8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:04:32.544 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 02:04:32 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x[62805]: 2024-03-19T02:04:32.142+0000 7fcf91e8c700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-19T02:04:32.544 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 02:04:32 smithi092 podman[121745]: 2024-03-19 02:04:32.174218035 +0000 UTC m=+0.161581158 container died 56cbeab0cd4c89214867b5094094ea0c63c8da43ccad7ea8a4a19ec66ca9b5b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-19T02:04:32.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 02:04:32 smithi092 podman[121745]: 2024-03-19 02:04:32.652443013 +0000 UTC m=+0.639806134 container cleanup 56cbeab0cd4c89214867b5094094ea0c63c8da43ccad7ea8a4a19ec66ca9b5b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b6ae24918b03b1b8a19c9263a50655bba80bb810, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-19T02:04:32.810 INFO:journalctl@ceph.mgr.x.smithi092.stdout:Mar 19 02:04:32 smithi092 bash[121745]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-x 2024-03-19T02:04:33.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-19T02:04:33.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:04:33.162 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-19T02:04:33.162 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-19T02:04:33.162 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.y 2024-03-19T02:04:33.484 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 02:04:33 smithi144 systemd[1]: Stopping Ceph mgr.y for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:04:33.797 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 02:04:33 smithi144 podman[103275]: 2024-03-19 02:04:33.484920765 +0000 UTC m=+0.128931376 container died fbebc8f9da7948db351f533eff3694551f768f42be998d8b7e73ef634c09a440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-03-19T02:04:34.096 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 02:04:33 smithi144 podman[103275]: 2024-03-19 02:04:33.90571659 +0000 UTC m=+0.549727191 container cleanup fbebc8f9da7948db351f533eff3694551f768f42be998d8b7e73ef634c09a440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-19T02:04:34.097 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 02:04:33 smithi144 bash[103275]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y 2024-03-19T02:04:34.376 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-19T02:04:34.400 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 02:04:34 smithi144 podman[103289]: 2024-03-19 02:04:34.149171294 +0000 UTC m=+0.660064684 container remove fbebc8f9da7948db351f533eff3694551f768f42be998d8b7e73ef634c09a440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-y, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:04:34.400 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 02:04:34 smithi144 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-19T02:04:34.400 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 02:04:34 smithi144 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.y.service: Failed with result 'exit-code'. 2024-03-19T02:04:34.400 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 02:04:34 smithi144 systemd[1]: Stopped Ceph mgr.y for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T02:04:34.400 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Mar 19 02:04:34 smithi144 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.y.service: Consumed 11.365s CPU time. 2024-03-19T02:04:34.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:04:34.413 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-19T02:04:34.413 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-19T02:04:34.413 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.z 2024-03-19T02:04:34.705 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 02:04:34 smithi159 systemd[1]: Stopping Ceph mgr.z for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:04:35.194 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 02:04:34 smithi159 podman[103260]: 2024-03-19 02:04:34.74655793 +0000 UTC m=+0.147104069 container died 259c300c508e5f0962f03c3202de3151e822ddde0f2f492032dd2074f84fc8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-03-19T02:04:35.455 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 02:04:35 smithi159 podman[103260]: 2024-03-19 02:04:35.302504608 +0000 UTC m=+0.703050748 container cleanup 259c300c508e5f0962f03c3202de3151e822ddde0f2f492032dd2074f84fc8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS) 2024-03-19T02:04:35.455 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 02:04:35 smithi159 bash[103260]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z 2024-03-19T02:04:35.730 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-19T02:04:35.753 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 02:04:35 smithi159 podman[103274]: 2024-03-19 02:04:35.495762994 +0000 UTC m=+0.744678998 container remove 259c300c508e5f0962f03c3202de3151e822ddde0f2f492032dd2074f84fc8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-19T02:04:35.754 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 02:04:35 smithi159 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-19T02:04:35.754 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 02:04:35 smithi159 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-03-19T02:04:35.754 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 02:04:35 smithi159 systemd[1]: Stopped Ceph mgr.z for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T02:04:35.754 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Mar 19 02:04:35 smithi159 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mgr.z.service: Consumed 11.238s CPU time. 2024-03-19T02:04:35.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:04:35.767 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-19T02:04:35.767 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-19T02:04:35.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.0 2024-03-19T02:04:36.060 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:35 smithi092 systemd[1]: Stopping Ceph osd.0 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:04:36.506 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0[77338]: 2024-03-19T02:04:36.079+0000 7f84d8422700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:04:36.506 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0[77338]: 2024-03-19T02:04:36.079+0000 7f84d8422700 -1 osd.0 90 *** Got signal Terminated *** 2024-03-19T02:04:36.506 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0[77338]: 2024-03-19T02:04:36.079+0000 7f84d8422700 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:04:41.722 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:41 smithi092 podman[121868]: 2024-03-19 02:04:41.405433974 +0000 UTC m=+5.453195799 container died e8aa00aea4b2f0a51e7ce9837d8f0e4655d369a1d85afecaf3d506af83f9a38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:04:42.060 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:41 smithi092 podman[121868]: 2024-03-19 02:04:41.804869328 +0000 UTC m=+5.852631155 container cleanup e8aa00aea4b2f0a51e7ce9837d8f0e4655d369a1d85afecaf3d506af83f9a38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-19T02:04:42.060 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:41 smithi092 bash[121868]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0 2024-03-19T02:04:42.329 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:42 smithi092 podman[121886]: 2024-03-19 02:04:42.064503026 +0000 UTC m=+0.657284624 container remove e8aa00aea4b2f0a51e7ce9837d8f0e4655d369a1d85afecaf3d506af83f9a38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-19T02:04:42.587 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:42 smithi092 podman[121976]: 2024-03-19 02:04:42.312977476 +0000 UTC m=+0.017606113 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:04:42.587 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:42 smithi092 podman[121976]: 2024-03-19 02:04:42.440755913 +0000 UTC m=+0.145384537 container create 8e2de95103e21aed7394dd19535e2cd294bb6e17be4df6a5ea0107e4ea291391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:04:43.029 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:42 smithi092 podman[121976]: 2024-03-19 02:04:42.735795928 +0000 UTC m=+0.440424562 container init 8e2de95103e21aed7394dd19535e2cd294bb6e17be4df6a5ea0107e4ea291391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:04:43.029 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:42 smithi092 podman[121976]: 2024-03-19 02:04:42.739327613 +0000 UTC m=+0.443956248 container start 8e2de95103e21aed7394dd19535e2cd294bb6e17be4df6a5ea0107e4ea291391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-19T02:04:43.029 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:42 smithi092 podman[121976]: 2024-03-19 02:04:42.835308363 +0000 UTC m=+0.539937003 container attach 8e2de95103e21aed7394dd19535e2cd294bb6e17be4df6a5ea0107e4ea291391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-19T02:04:43.310 INFO:journalctl@ceph.osd.0.smithi092.stdout:Mar 19 02:04:43 smithi092 podman[121976]: 2024-03-19 02:04:43.029713227 +0000 UTC m=+0.734341845 container died 8e2de95103e21aed7394dd19535e2cd294bb6e17be4df6a5ea0107e4ea291391 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-19T02:04:43.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.0.service' 2024-03-19T02:04:43.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:04:43.772 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-19T02:04:43.772 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-19T02:04:43.772 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.1 2024-03-19T02:04:44.093 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:43 smithi144 systemd[1]: Stopping Ceph osd.1 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:04:44.346 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1[71782]: 2024-03-19T02:04:44.092+0000 7eff06e96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:04:44.346 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1[71782]: 2024-03-19T02:04:44.092+0000 7eff06e96700 -1 osd.1 90 *** Got signal Terminated *** 2024-03-19T02:04:44.346 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1[71782]: 2024-03-19T02:04:44.092+0000 7eff06e96700 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:04:49.752 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:49 smithi144 podman[103399]: 2024-03-19 02:04:49.426918381 +0000 UTC m=+5.450910261 container died 17d3c096fe1da47a0e7bb6d3fb4df72d409de8fdaf72105ba3023597318a0174 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-03-19T02:04:50.097 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:49 smithi144 podman[103399]: 2024-03-19 02:04:49.834649648 +0000 UTC m=+5.858641526 container cleanup 17d3c096fe1da47a0e7bb6d3fb4df72d409de8fdaf72105ba3023597318a0174 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True) 2024-03-19T02:04:50.097 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:49 smithi144 bash[103399]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1 2024-03-19T02:04:50.559 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:50 smithi144 podman[103416]: 2024-03-19 02:04:50.119959707 +0000 UTC m=+0.692391200 container remove 17d3c096fe1da47a0e7bb6d3fb4df72d409de8fdaf72105ba3023597318a0174 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.build-date=20240307) 2024-03-19T02:04:50.846 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:50 smithi144 podman[103505]: 2024-03-19 02:04:50.462254694 +0000 UTC m=+0.017913437 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:04:50.847 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:50 smithi144 podman[103505]: 2024-03-19 02:04:50.637859643 +0000 UTC m=+0.193518366 container create cae35ca6ba0034ac4a58851d64898b1d4f59fa26dddc19cf79fed465fa270449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T02:04:51.213 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:50 smithi144 podman[103505]: 2024-03-19 02:04:50.916006469 +0000 UTC m=+0.471665201 container init cae35ca6ba0034ac4a58851d64898b1d4f59fa26dddc19cf79fed465fa270449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-19T02:04:51.213 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:50 smithi144 podman[103505]: 2024-03-19 02:04:50.919266002 +0000 UTC m=+0.474924733 container start cae35ca6ba0034ac4a58851d64898b1d4f59fa26dddc19cf79fed465fa270449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-19T02:04:51.213 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:50 smithi144 podman[103505]: 2024-03-19 02:04:50.990433591 +0000 UTC m=+0.546092335 container attach cae35ca6ba0034ac4a58851d64898b1d4f59fa26dddc19cf79fed465fa270449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-19T02:04:51.478 INFO:journalctl@ceph.osd.1.smithi144.stdout:Mar 19 02:04:51 smithi144 podman[103505]: 2024-03-19 02:04:51.21411201 +0000 UTC m=+0.769770753 container died cae35ca6ba0034ac4a58851d64898b1d4f59fa26dddc19cf79fed465fa270449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-1-deactivate, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-19T02:04:51.904 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.1.service' 2024-03-19T02:04:51.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:04:51.945 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-19T02:04:51.945 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-19T02:04:51.945 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.2 2024-03-19T02:04:52.256 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:51 smithi159 systemd[1]: Stopping Ceph osd.2 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:04:52.707 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:52 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2[72251]: 2024-03-19T02:04:52.256+0000 7fa47c9dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:04:52.707 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:52 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2[72251]: 2024-03-19T02:04:52.256+0000 7fa47c9dc700 -1 osd.2 90 *** Got signal Terminated *** 2024-03-19T02:04:52.707 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:52 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2[72251]: 2024-03-19T02:04:52.256+0000 7fa47c9dc700 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:04:57.948 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:57 smithi159 podman[103385]: 2024-03-19 02:04:57.678817521 +0000 UTC m=+5.540444523 container died b15129fcee2fe67bf0c8bc2be4901cd8fb8dd69203d66811a2e688a564f6460f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T02:04:58.205 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:58 smithi159 podman[103385]: 2024-03-19 02:04:58.04837551 +0000 UTC m=+5.910002503 container cleanup b15129fcee2fe67bf0c8bc2be4901cd8fb8dd69203d66811a2e688a564f6460f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:04:58.205 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:58 smithi159 bash[103385]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2 2024-03-19T02:04:58.656 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:58 smithi159 podman[103403]: 2024-03-19 02:04:58.341548662 +0000 UTC m=+0.656646108 container remove b15129fcee2fe67bf0c8bc2be4901cd8fb8dd69203d66811a2e688a564f6460f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307) 2024-03-19T02:04:58.920 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:58 smithi159 podman[103496]: 2024-03-19 02:04:58.559909821 +0000 UTC m=+0.018271615 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:04:58.920 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:58 smithi159 podman[103496]: 2024-03-19 02:04:58.684340368 +0000 UTC m=+0.142702148 container create 5ca25205e7872e4739c77b2c9e3917ac6ca0b23a5e8377aa7f5d9a719e108500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-19T02:04:59.205 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:58 smithi159 podman[103496]: 2024-03-19 02:04:58.920609628 +0000 UTC m=+0.378971411 container init 5ca25205e7872e4739c77b2c9e3917ac6ca0b23a5e8377aa7f5d9a719e108500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:04:59.205 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:58 smithi159 podman[103496]: 2024-03-19 02:04:58.923805343 +0000 UTC m=+0.382167121 container start 5ca25205e7872e4739c77b2c9e3917ac6ca0b23a5e8377aa7f5d9a719e108500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-03-19T02:04:59.206 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:59 smithi159 podman[103496]: 2024-03-19 02:04:59.020599284 +0000 UTC m=+0.478961081 container attach 5ca25205e7872e4739c77b2c9e3917ac6ca0b23a5e8377aa7f5d9a719e108500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-19T02:04:59.479 INFO:journalctl@ceph.osd.2.smithi159.stdout:Mar 19 02:04:59 smithi159 podman[103496]: 2024-03-19 02:04:59.210340564 +0000 UTC m=+0.668702357 container died 5ca25205e7872e4739c77b2c9e3917ac6ca0b23a5e8377aa7f5d9a719e108500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-19T02:04:59.855 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.2.service' 2024-03-19T02:04:59.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:04:59.896 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-19T02:04:59.897 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-19T02:04:59.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.3 2024-03-19T02:05:00.224 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:04:59 smithi092 systemd[1]: Stopping Ceph osd.3 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:05:00.482 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:00 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3[83857]: 2024-03-19T02:05:00.223+0000 7fec7adef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:05:00.482 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:00 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3[83857]: 2024-03-19T02:05:00.223+0000 7fec7adef700 -1 osd.3 90 *** Got signal Terminated *** 2024-03-19T02:05:00.482 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:00 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3[83857]: 2024-03-19T02:05:00.223+0000 7fec7adef700 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:05:01.810 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:01 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3[83857]: 2024-03-19T02:05:01.649+0000 7fec773e3700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:38.622468+0000 front 2024-03-19T02:04:38.622413+0000 (oldest deadline 2024-03-19T02:05:01.522714+0000) 2024-03-19T02:05:03.006 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:02 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3[83857]: 2024-03-19T02:05:02.665+0000 7fec773e3700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:38.622468+0000 front 2024-03-19T02:04:38.622413+0000 (oldest deadline 2024-03-19T02:05:01.522714+0000) 2024-03-19T02:05:03.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:03 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:03.156+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:03.313 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:03 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:03.006+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:03.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:03.027+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:03.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:03.119+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:04.026 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:03 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3[83857]: 2024-03-19T02:05:03.624+0000 7fec773e3700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:38.622468+0000 front 2024-03-19T02:04:38.622413+0000 (oldest deadline 2024-03-19T02:05:01.522714+0000) 2024-03-19T02:05:04.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:03 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:03.813+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:04.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:04 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:04.025+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:04.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:04 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:04.126+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:04.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:04 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:04.018+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:04.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:04 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:04.078+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:05.033 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:04 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3[83857]: 2024-03-19T02:05:04.579+0000 7fec773e3700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:38.622468+0000 front 2024-03-19T02:04:38.622413+0000 (oldest deadline 2024-03-19T02:05:01.522714+0000) 2024-03-19T02:05:05.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:04 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:04.810+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:05.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:05 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:05.033+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:05.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:05 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:05.086+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:05.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:05.063+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:05.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:05.098+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:05.949 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:05 smithi092 podman[122084]: 2024-03-19 02:05:05.618271338 +0000 UTC m=+5.523578402 container died 22301136b211d47149f72b7c9f8f9dd3f098733f121f5f1eeec773867269cb6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:05:06.069 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:05.798+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:06.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:05 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:05.808+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:06.274 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:05 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:05.995+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:06.275 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:06 smithi092 podman[122084]: 2024-03-19 02:05:06.065667977 +0000 UTC m=+5.970975035 container cleanup 22301136b211d47149f72b7c9f8f9dd3f098733f121f5f1eeec773867269cb6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2) 2024-03-19T02:05:06.275 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:06 smithi092 bash[122084]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3 2024-03-19T02:05:06.275 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:06 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:06.039+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:06.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:06.069+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:06.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:06.110+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:06.560 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:06 smithi092 podman[122113]: 2024-03-19 02:05:06.27541005 +0000 UTC m=+0.656818030 container remove 22301136b211d47149f72b7c9f8f9dd3f098733f121f5f1eeec773867269cb6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-19T02:05:06.945 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:06 smithi092 podman[122208]: 2024-03-19 02:05:06.578989286 +0000 UTC m=+0.018087285 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:05:06.945 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:06 smithi092 podman[122208]: 2024-03-19 02:05:06.69312022 +0000 UTC m=+0.132218245 container create bb7fa93d93e5226409a0a8bc9710b7c59c0cc7968031b364f2f9c2cac5b08a75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:05:06.945 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:06 smithi092 podman[122208]: 2024-03-19 02:05:06.945550053 +0000 UTC m=+0.384648043 container init bb7fa93d93e5226409a0a8bc9710b7c59c0cc7968031b364f2f9c2cac5b08a75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-19T02:05:07.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:06 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:06.841+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:07.107 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:07.063+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:07.107 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:06.822+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:07.244 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:07 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:07.057+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:07.244 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:06 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:06.974+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:07.244 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:06 smithi092 podman[122208]: 2024-03-19 02:05:06.948739208 +0000 UTC m=+0.387837192 container start bb7fa93d93e5226409a0a8bc9710b7c59c0cc7968031b364f2f9c2cac5b08a75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-19T02:05:07.245 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:07 smithi092 podman[122208]: 2024-03-19 02:05:07.054008687 +0000 UTC m=+0.493106673 container attach bb7fa93d93e5226409a0a8bc9710b7c59c0cc7968031b364f2f9c2cac5b08a75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, ceph=True) 2024-03-19T02:05:07.245 INFO:journalctl@ceph.osd.3.smithi092.stdout:Mar 19 02:05:07 smithi092 podman[122208]: 2024-03-19 02:05:07.244688558 +0000 UTC m=+0.683786542 container died bb7fa93d93e5226409a0a8bc9710b7c59c0cc7968031b364f2f9c2cac5b08a75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:05:07.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:07.107+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:07.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.3.service' 2024-03-19T02:05:07.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:05:07.976 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-19T02:05:07.977 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-19T02:05:07.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.4 2024-03-19T02:05:08.051 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:07.779+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:08.297 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:08 smithi144 systemd[1]: Stopping Ceph osd.4 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:05:08.297 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:07 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:07.884+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:08.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:08 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:08.015+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:08.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:07 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:07.956+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:08.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:08.096+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:08.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:08.051+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:08.596 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:08 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4[78733]: 2024-03-19T02:05:08.296+0000 7f0fa36ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:05:08.596 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:08 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4[78733]: 2024-03-19T02:05:08.296+0000 7f0fa36ba700 -1 osd.4 90 *** Got signal Terminated *** 2024-03-19T02:05:08.597 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:08 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4[78733]: 2024-03-19T02:05:08.296+0000 7f0fa36ba700 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:05:09.019 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:08.735+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:09.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:08 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:08.958+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:09.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:09 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:09.041+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:09.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:08 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:08.908+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:09.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:09 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:09.094+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:09.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:09 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:09.019+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:10.060 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:09 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:09.729+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:10.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:09 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:09.936+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:10.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:10 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:10.039+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:10.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:09 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:09.883+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:10.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:10 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:10.069+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:10.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:10 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:10.060+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:10.839 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:10 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:10.391+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:10.959 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:10 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:10.702+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:10.959 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:10 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:10.702+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:11.098 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:10 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:10.837+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:11.098 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:10 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:10.837+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:11.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:10 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:10.981+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:11.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:10 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:10.998+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:11.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:10 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:10.998+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:11.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:11 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:11.014+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:11.456 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:11 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:11.069+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:11.836 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:11 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:11.412+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:11.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:11 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:11.658+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:11.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:11 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:11.658+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:12.097 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:11 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:11.835+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:12.097 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:11 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:11.835+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:12.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:12 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:12.022+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:12.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:11 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:11.965+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:12.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:11 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:11.965+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:12.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:12 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:12.051+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:12.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:11 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:11.982+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:12.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:12 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:12.442+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:12.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:12 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:12.690+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:12.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:12 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:12.690+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:13.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:12 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:12.971+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:13.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:12 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:12.971+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:13.312 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:13 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:13.058+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:13.312 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:13 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:13.058+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:13.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:12 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:12.854+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:13.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:12 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:12.854+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:13.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:13 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:13.095+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:13.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:13 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:13.019+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:13.736 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:13 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:13.458+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:14.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:13 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:13.728+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:14.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:13 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:13.728+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:14.055 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:13 smithi144 podman[103623]: 2024-03-19 02:05:13.736650457 +0000 UTC m=+5.566545400 container died 6f15b84e1e269869628f537407f855b0166e1411cd8d900ab56e96eab9a35016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:05:14.055 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:13 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:13.867+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:14.055 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:13 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:13.867+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:14.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:13 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:13.998+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:14.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:13 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:13.998+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:14.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:14 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:14.098+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:14.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:14 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:14.098+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:14.346 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:14 smithi144 podman[103623]: 2024-03-19 02:05:14.163803391 +0000 UTC m=+5.993698325 container cleanup 6f15b84e1e269869628f537407f855b0166e1411cd8d900ab56e96eab9a35016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-19T02:05:14.347 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:14 smithi144 bash[103623]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4 2024-03-19T02:05:14.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:14 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:14.086+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:14.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:14 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:14.086+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.067953+0000 front 2024-03-19T02:04:48.068062+0000 (oldest deadline 2024-03-19T02:05:13.367953+0000) 2024-03-19T02:05:14.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:14 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:14.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:14.767 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:14 smithi144 podman[103641]: 2024-03-19 02:05:14.365016928 +0000 UTC m=+0.624065453 container remove 6f15b84e1e269869628f537407f855b0166e1411cd8d900ab56e96eab9a35016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:05:14.767 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:14 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:14.481+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:15.018 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:14 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:14.829+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:15.019 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:14 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:14.829+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:15.019 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:14 smithi144 podman[103729]: 2024-03-19 02:05:14.670336616 +0000 UTC m=+0.017763965 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:05:15.019 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:14 smithi144 podman[103729]: 2024-03-19 02:05:14.774310218 +0000 UTC m=+0.121737549 container create 047c83267f8f23ac54fc478a9ba635ad7d1c2eb68a560ea34ee64fb22fbd3626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-19T02:05:15.092 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:14 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:14.770+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:15.092 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:14 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:14.770+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:15.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:14 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:14.970+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:15.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:14 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:14.970+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:15.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:15 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:15.107+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:15.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:15 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:15.107+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:15.311 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:15 smithi144 podman[103729]: 2024-03-19 02:05:15.019388851 +0000 UTC m=+0.366816196 container init 047c83267f8f23ac54fc478a9ba635ad7d1c2eb68a560ea34ee64fb22fbd3626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-19T02:05:15.311 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:15 smithi144 podman[103729]: 2024-03-19 02:05:15.022704272 +0000 UTC m=+0.370131605 container start 047c83267f8f23ac54fc478a9ba635ad7d1c2eb68a560ea34ee64fb22fbd3626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-19T02:05:15.311 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:15 smithi144 podman[103729]: 2024-03-19 02:05:15.110318074 +0000 UTC m=+0.457745424 container attach 047c83267f8f23ac54fc478a9ba635ad7d1c2eb68a560ea34ee64fb22fbd3626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-19T02:05:15.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:15 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:15.111+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:15.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:15 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:15.111+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.067953+0000 front 2024-03-19T02:04:48.068062+0000 (oldest deadline 2024-03-19T02:05:13.367953+0000) 2024-03-19T02:05:15.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:15 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:15.092+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:15.596 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:15 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:15.528+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:15.597 INFO:journalctl@ceph.osd.4.smithi144.stdout:Mar 19 02:05:15 smithi144 podman[103729]: 2024-03-19 02:05:15.311549939 +0000 UTC m=+0.658977268 container died 047c83267f8f23ac54fc478a9ba635ad7d1c2eb68a560ea34ee64fb22fbd3626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-03-19T02:05:15.963 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:15 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:15.834+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:15.963 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:15 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:15.834+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:15.982 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.4.service' 2024-03-19T02:05:16.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:05:16.024 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-19T02:05:16.024 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-19T02:05:16.024 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.5 2024-03-19T02:05:16.053 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:15 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:15.800+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:16.053 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:15 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:15.800+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:16.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:16 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:16.008+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:16.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:16 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:16.008+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:16.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:16 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:16.131+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:16.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:16 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:16.131+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:16.335 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:16 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:16.100+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:16.335 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:16 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:16.067+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:16.335 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:16 smithi159 systemd[1]: Stopping Ceph osd.5 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:05:16.335 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:16 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:16.067+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.067953+0000 front 2024-03-19T02:04:48.068062+0000 (oldest deadline 2024-03-19T02:05:13.367953+0000) 2024-03-19T02:05:16.705 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:16 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:16.334+0000 7fce8d6cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:05:16.705 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:16 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:16.334+0000 7fce8d6cc700 -1 osd.5 90 *** Got signal Terminated *** 2024-03-19T02:05:16.705 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:16 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:16.334+0000 7fce8d6cc700 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:05:16.836 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:16 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:16.479+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:17.057 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:16 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:16.763+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:17.058 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:16 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:16.763+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:17.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:16 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:16.836+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:17.097 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:16 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:16.836+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:17.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:16 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:16.972+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:17.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:16 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:16.972+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:17.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:17.113+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:17.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:17.113+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:17.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:17 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:17.057+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:17.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:17 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:17.057+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.067953+0000 front 2024-03-19T02:04:48.068062+0000 (oldest deadline 2024-03-19T02:05:13.367953+0000) 2024-03-19T02:05:17.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:17 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:17.060+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:17.834 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:17 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:17.469+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:18.078 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:17 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:17.720+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:18.078 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:17 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:17.720+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:18.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:17 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:17.834+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:18.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:17 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:17.834+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:18.097 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:17 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:17.834+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:18.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:18 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:18.010+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:18.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:18 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:18.010+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:18.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:18 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:18.144+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:18.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:18 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:18.144+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:18.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:18 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:18.078+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:18.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:18 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:18.078+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.067953+0000 front 2024-03-19T02:04:48.068062+0000 (oldest deadline 2024-03-19T02:05:13.367953+0000) 2024-03-19T02:05:18.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:18 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:18.084+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:18.833 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:18 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:18.431+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:18.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:18 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:18.675+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:18.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:18 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:18.675+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:18.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:18 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:18.675+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:19.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:18 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:18.834+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:19.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:18 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:18.834+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:19.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:18 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:18.834+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:19.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:19 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:19.022+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:19.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:19 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:19.022+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:19.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:19 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:19.174+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:19.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:19 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:19.175+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:19.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:19 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:19.175+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:19.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:19 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:19.112+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:19.456 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:19 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:19.053+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:19.456 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:19 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:19.053+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.067953+0000 front 2024-03-19T02:04:48.068062+0000 (oldest deadline 2024-03-19T02:05:13.367953+0000) 2024-03-19T02:05:19.840 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:19 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:19.395+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:19.840 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:19 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:19.395+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:19.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:19 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:19.686+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:19.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:19 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:19.686+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:19.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:19 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:19.686+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:20.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:19 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:19.840+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:20.096 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:19 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:19.840+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:20.097 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:19 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:19.840+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:20.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:20 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:20.025+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:20.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:20 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:20.025+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:20.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:20 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:20.194+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:20.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:20 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:20.194+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:20.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:20 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:20.194+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:20.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:20 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:20.065+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:20.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:20 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:20.065+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.067953+0000 front 2024-03-19T02:04:48.068062+0000 (oldest deadline 2024-03-19T02:05:13.367953+0000) 2024-03-19T02:05:20.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:20 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:20.068+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:20.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:20 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:20.438+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:20.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:20 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:20.438+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:21.046 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:20 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:20.706+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:21.046 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:20 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:20.706+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:21.046 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:20 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:20.706+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:21.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:21 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:21.055+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:21.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:21 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:21.055+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:21.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:21 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:21.055+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:54.464609+0000 front 2024-03-19T02:04:54.464596+0000 (oldest deadline 2024-03-19T02:05:20.364581+0000) 2024-03-19T02:05:21.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:21 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:21.233+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:21.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:21 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:21.233+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:21.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:21 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:21.233+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:21.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:20 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:20.874+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:21.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:20 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:20.874+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:21.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:20 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:20.874+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:21.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:21 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:21.071+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:21.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:21 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:21.045+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:35.365366+0000 front 2024-03-19T02:04:35.365190+0000 (oldest deadline 2024-03-19T02:05:02.365616+0000) 2024-03-19T02:05:21.455 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:21 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:21.045+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.067953+0000 front 2024-03-19T02:04:48.068062+0000 (oldest deadline 2024-03-19T02:05:13.367953+0000) 2024-03-19T02:05:21.456 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:21 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5[79204]: 2024-03-19T02:05:21.045+0000 7fce89cc0700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.368330+0000 front 2024-03-19T02:04:53.368296+0000 (oldest deadline 2024-03-19T02:05:20.969105+0000) 2024-03-19T02:05:21.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:21 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:21.475+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:21.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:21 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:21.475+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:22.027 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:21 smithi159 podman[103613]: 2024-03-19 02:05:21.741806275 +0000 UTC m=+5.530455112 container died 47f662b193e1c7b62bdfd843e9a70314923e5b173d3f08c3c16a574e15afa50a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-19T02:05:22.028 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:21 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:21.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:22.028 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:21 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:21.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:22.028 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:21 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:21.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:22.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:22 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:22.027+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:22.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:22 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:22.027+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:22.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:22 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:22.027+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:54.464609+0000 front 2024-03-19T02:04:54.464596+0000 (oldest deadline 2024-03-19T02:05:20.364581+0000) 2024-03-19T02:05:22.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:22 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:22.221+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:22.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:22 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:22.221+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:22.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:22 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:22.221+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:22.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:21 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:21.871+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:22.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:21 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:21.871+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:22.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:21 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:21.871+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:22.377 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:22 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:22.034+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:22.378 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:22 smithi159 podman[103613]: 2024-03-19 02:05:22.135554371 +0000 UTC m=+5.924203197 container cleanup 47f662b193e1c7b62bdfd843e9a70314923e5b173d3f08c3c16a574e15afa50a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:05:22.378 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:22 smithi159 bash[103613]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5 2024-03-19T02:05:22.692 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:22 smithi159 podman[103630]: 2024-03-19 02:05:22.378608801 +0000 UTC m=+0.633089202 container remove 47f662b193e1c7b62bdfd843e9a70314923e5b173d3f08c3c16a574e15afa50a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-19T02:05:22.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:22 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:22.524+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:22.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:22 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:22.524+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:22.956 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:22 smithi159 podman[103720]: 2024-03-19 02:05:22.596003675 +0000 UTC m=+0.017999252 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:05:22.956 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:22 smithi159 podman[103720]: 2024-03-19 02:05:22.721187559 +0000 UTC m=+0.143183236 container create 99f34f7fbc031a53fbf0f4b16907b236e28beaff6d3bced95dfeca9878fb52ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-19T02:05:22.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:22 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:22.740+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:22.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:22 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:22.740+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:22.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:22 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:22.740+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:23.277 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:23 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:23.072+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:23.277 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:22 smithi159 podman[103720]: 2024-03-19 02:05:22.98236551 +0000 UTC m=+0.404361073 container init 99f34f7fbc031a53fbf0f4b16907b236e28beaff6d3bced95dfeca9878fb52ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-19T02:05:23.277 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:22 smithi159 podman[103720]: 2024-03-19 02:05:22.985634485 +0000 UTC m=+0.407630058 container start 99f34f7fbc031a53fbf0f4b16907b236e28beaff6d3bced95dfeca9878fb52ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307) 2024-03-19T02:05:23.277 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:23 smithi159 podman[103720]: 2024-03-19 02:05:23.082419375 +0000 UTC m=+0.504414949 container attach 99f34f7fbc031a53fbf0f4b16907b236e28beaff6d3bced95dfeca9878fb52ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:05:23.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:23 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:23.176+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:23.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:23 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:23.176+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:23.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:23 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:23.176+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:23.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:23 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:23.058+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:23.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:23 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:23.058+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:23.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:23 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:23.058+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:54.464609+0000 front 2024-03-19T02:04:54.464596+0000 (oldest deadline 2024-03-19T02:05:20.364581+0000) 2024-03-19T02:05:23.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:22 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:22.896+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:23.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:22 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:22.896+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:23.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:22 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:22.896+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:23.577 INFO:journalctl@ceph.osd.5.smithi159.stdout:Mar 19 02:05:23 smithi159 podman[103720]: 2024-03-19 02:05:23.277814365 +0000 UTC m=+0.699809939 container died 99f34f7fbc031a53fbf0f4b16907b236e28beaff6d3bced95dfeca9878fb52ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-03-19T02:05:23.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:23 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:23.510+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:23.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:23 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:23.510+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:23.885 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:23 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:23.756+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:23.885 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:23 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:23.756+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:23.886 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:23 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:23.756+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:23.906 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.5.service' 2024-03-19T02:05:23.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:05:23.988 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-19T02:05:23.988 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-19T02:05:23.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.6 2024-03-19T02:05:24.205 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:24.044+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:24.205 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:24.044+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:24.309 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:24.015+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:24.309 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:24.015+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:24.309 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:24.015+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:54.464609+0000 front 2024-03-19T02:04:54.464596+0000 (oldest deadline 2024-03-19T02:05:20.364581+0000) 2024-03-19T02:05:24.309 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:24 smithi092 systemd[1]: Stopping Ceph osd.6 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:05:24.309 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:24.176+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:24.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:24.176+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:24.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:24.176+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:24.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:23 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:23.846+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:24.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:23 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:23.846+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:24.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:23 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:23.846+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:24.560 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:24.308+0000 7fd1a4c7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:05:24.560 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:24.308+0000 7fd1a4c7e700 -1 osd.6 90 *** Got signal Terminated *** 2024-03-19T02:05:24.560 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:24 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:24.308+0000 7fd1a4c7e700 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:05:24.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:24.473+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:24.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:24.473+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:25.205 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:25 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:25.011+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:25.205 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:25 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:25.011+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:25.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:24.782+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:25.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:24.782+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:25.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:24 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:24.782+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:25.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:24.894+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:25.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:24.894+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:25.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:24 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:24.894+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:25.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:25 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:25.221+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:25.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:25 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:25.221+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:25.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:25 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:25.221+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:25.560 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:25 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:25.062+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:25.561 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:25 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:25.062+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:25.561 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:25 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:25.062+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:54.464609+0000 front 2024-03-19T02:04:54.464596+0000 (oldest deadline 2024-03-19T02:05:20.364581+0000) 2024-03-19T02:05:25.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:25 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:25.517+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:25.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:25 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:25.517+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:26.025 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:25 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:25.760+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:26.025 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:25 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:25.760+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:26.025 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:25 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:25.760+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:26.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:26 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:26.202+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:26.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:26 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:26.202+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:26.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:26 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:26.202+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:26.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:26 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:26.030+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:26.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:26 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:26.030+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:26.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:26 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:26.030+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:54.464609+0000 front 2024-03-19T02:04:54.464596+0000 (oldest deadline 2024-03-19T02:05:20.364581+0000) 2024-03-19T02:05:26.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:25 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:25.897+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:26.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:25 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:25.897+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:26.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:25 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:25.897+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:26.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:25 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:25.897+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:26.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:26 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:26.025+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:26.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:26 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:26.025+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:26.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:26 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:26.521+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:26.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:26 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:26.521+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:27.039 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:26 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:26.765+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:27.040 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:26 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:26.765+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:27.040 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:26 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:26.765+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:27.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:27.201+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:27.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:27.201+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:27.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:27.201+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:27.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:27.202+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:27.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:27.041+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:27.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:27.041+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:27.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:27.041+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:54.464609+0000 front 2024-03-19T02:04:54.464596+0000 (oldest deadline 2024-03-19T02:05:20.364581+0000) 2024-03-19T02:05:27.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:27.041+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:02.665468+0000 front 2024-03-19T02:05:02.665380+0000 (oldest deadline 2024-03-19T02:05:26.765515+0000) 2024-03-19T02:05:27.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:26 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:26.933+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:27.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:26 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:26.933+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:27.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:26 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:26.933+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:27.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:26 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:26.933+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:27.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:27 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:27.039+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:27.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:27 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:27.039+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:27.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:27 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:27.039+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:27.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:27 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:27.495+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:27.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:27 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:27.495+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:28.053 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:27 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:27.750+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:28.053 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:27 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:27.750+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:28.053 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:27 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:27.750+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:28.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:27.998+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:28.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:27.998+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:28.310 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:27.998+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:54.464609+0000 front 2024-03-19T02:04:54.464596+0000 (oldest deadline 2024-03-19T02:05:20.364581+0000) 2024-03-19T02:05:28.311 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:27 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:27.998+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:02.665468+0000 front 2024-03-19T02:05:02.665380+0000 (oldest deadline 2024-03-19T02:05:26.765515+0000) 2024-03-19T02:05:28.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:28.239+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:28.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:28.239+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:28.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:28.239+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:28.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:28.239+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:28.311 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:28 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:28.072+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:28.311 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:28 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:28.072+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:28.311 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:28 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:28.072+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:28.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:27 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:27.943+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:28.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:27 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:27.943+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:28.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:27 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:27.943+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:28.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:27 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:27.943+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:28.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:28 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:28.451+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:28.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:28 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:28.451+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:29.036 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:28 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:28.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:29.036 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:28 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:28.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:29.036 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:28 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:28.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:29.036 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:28 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:28.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:29.283 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:28.992+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.362373+0000 front 2024-03-19T02:04:39.362171+0000 (oldest deadline 2024-03-19T02:05:02.262279+0000) 2024-03-19T02:05:29.283 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:28.992+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.663600+0000 front 2024-03-19T02:04:48.663627+0000 (oldest deadline 2024-03-19T02:05:10.363731+0000) 2024-03-19T02:05:29.283 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:28.992+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:54.464609+0000 front 2024-03-19T02:04:54.464596+0000 (oldest deadline 2024-03-19T02:05:20.364581+0000) 2024-03-19T02:05:29.283 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:28 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6[90829]: 2024-03-19T02:05:28.992+0000 7fd19d26a700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:02.665468+0000 front 2024-03-19T02:05:02.665380+0000 (oldest deadline 2024-03-19T02:05:26.765515+0000) 2024-03-19T02:05:29.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:28 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:28.894+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:29.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:28 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:28.894+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:29.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:28 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:28.894+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:29.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:28 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:28.894+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:29.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:29 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:29.035+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:29.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:29 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:29.035+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:29.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:29 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:29.035+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:29.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:29 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:29.283+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:29.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:29 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:29.283+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:29.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:29 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:29.283+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:29.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:29 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:29.283+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:29.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:29 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:29.432+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:29.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:29 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:29.432+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:29.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:29 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:29.432+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.435816+0000 front 2024-03-19T02:05:03.435838+0000 (oldest deadline 2024-03-19T02:05:28.735708+0000) 2024-03-19T02:05:30.001 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:29 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:29.725+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:30.001 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:29 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:29.725+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:30.001 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:29 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:29.725+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:30.001 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:29 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:29.725+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:30.061 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:29 smithi092 podman[122314]: 2024-03-19 02:05:29.797760946 +0000 UTC m=+5.618532061 container died 621eab5e8a617b0952a729b6242238b1888cde071332f87af437a2556e020fe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:05:30.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:29 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:29.903+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:30.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:29 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:29.903+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:30.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:29 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:29.903+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:30.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:29 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:29.903+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:30.434 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:30 smithi092 podman[122314]: 2024-03-19 02:05:30.158764122 +0000 UTC m=+5.979535243 container cleanup 621eab5e8a617b0952a729b6242238b1888cde071332f87af437a2556e020fe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-19T02:05:30.434 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:30 smithi092 bash[122314]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6 2024-03-19T02:05:30.435 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:30 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:30.284+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:30.435 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:30 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:30.284+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:30.435 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:30 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:30.284+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:30.435 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:30 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:30.284+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:30.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:30 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:30.001+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:30.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:30 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:30.001+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:30.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:30 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:30.001+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:30.739 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:30 smithi092 podman[122331]: 2024-03-19 02:05:30.434971195 +0000 UTC m=+0.629677558 container remove 621eab5e8a617b0952a729b6242238b1888cde071332f87af437a2556e020fe3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-19T02:05:30.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:30 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:30.445+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:30.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:30 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:30.445+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:30.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:30 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:30.445+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.435816+0000 front 2024-03-19T02:05:03.435838+0000 (oldest deadline 2024-03-19T02:05:28.735708+0000) 2024-03-19T02:05:31.004 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:30 smithi092 podman[122421]: 2024-03-19 02:05:30.64311425 +0000 UTC m=+0.018146023 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:05:31.004 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:30 smithi092 podman[122421]: 2024-03-19 02:05:30.769453992 +0000 UTC m=+0.144485745 container create b1dc1685f557d4fc47301da7f44be2a012ee8dfa733a0155b07ac5bb6fb25310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-19T02:05:31.015 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:30 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:30.765+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:31.015 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:30 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:30.765+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:31.016 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:30 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:30.765+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:31.016 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:30 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:30.765+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:31.256 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:31 smithi092 podman[122421]: 2024-03-19 02:05:31.005159144 +0000 UTC m=+0.380190911 container init b1dc1685f557d4fc47301da7f44be2a012ee8dfa733a0155b07ac5bb6fb25310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-19T02:05:31.256 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:31 smithi092 podman[122421]: 2024-03-19 02:05:31.008988668 +0000 UTC m=+0.384020423 container start b1dc1685f557d4fc47301da7f44be2a012ee8dfa733a0155b07ac5bb6fb25310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD) 2024-03-19T02:05:31.256 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:31 smithi092 podman[122421]: 2024-03-19 02:05:31.105224488 +0000 UTC m=+0.480256247 container attach b1dc1685f557d4fc47301da7f44be2a012ee8dfa733a0155b07ac5bb6fb25310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:05:31.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:30 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:30.921+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:31.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:30 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:30.921+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:31.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:30 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:30.921+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:31.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:30 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:30.921+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:31.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:31 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:31.015+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:31.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:31 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:31.015+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:31.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:31 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:31.015+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:31.560 INFO:journalctl@ceph.osd.6.smithi092.stdout:Mar 19 02:05:31 smithi092 podman[122421]: 2024-03-19 02:05:31.295662423 +0000 UTC m=+0.670694187 container died b1dc1685f557d4fc47301da7f44be2a012ee8dfa733a0155b07ac5bb6fb25310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-19T02:05:31.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:31 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:31.255+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:31.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:31 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:31.255+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:31.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:31 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:31.255+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:31.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:31 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:31.255+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:31.619 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:31.490+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:31.619 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:31.490+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:31.619 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:31.490+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.435816+0000 front 2024-03-19T02:05:03.435838+0000 (oldest deadline 2024-03-19T02:05:28.735708+0000) 2024-03-19T02:05:31.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.6.service' 2024-03-19T02:05:31.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:05:31.978 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-19T02:05:31.978 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-19T02:05:31.978 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.7 2024-03-19T02:05:32.047 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:31 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:31.773+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:32.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:31 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:31.773+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:32.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:31 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:31.773+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:32.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:31 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:31.773+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:32.308 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:32 smithi144 systemd[1]: Stopping Ceph osd.7 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:05:32.308 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:31.933+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:32.308 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:31.933+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:32.309 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:31.933+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:32.309 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:31.933+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:32.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:32 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:32.046+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:32.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:32 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:32.046+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:32.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:32 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:32.046+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:32.497 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:32 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:32.227+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:32.497 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:32 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:32.227+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:32.497 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:32 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:32.227+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:32.497 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:32 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:32.227+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:32.566 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:32.307+0000 7f96c365c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:05:32.566 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:32.307+0000 7f96c365c700 -1 osd.7 90 *** Got signal Terminated *** 2024-03-19T02:05:32.567 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:32.307+0000 7f96c365c700 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:05:32.567 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:32.491+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:32.567 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:32.491+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:32.567 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:32.491+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.435816+0000 front 2024-03-19T02:05:03.435838+0000 (oldest deadline 2024-03-19T02:05:28.735708+0000) 2024-03-19T02:05:33.014 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:32 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:32.751+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:33.014 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:32 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:32.751+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:33.014 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:32 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:32.751+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:33.014 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:32 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:32.751+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:33.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:32.948+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:33.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:32.948+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:33.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:32.948+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:33.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:32 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:32.948+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:33.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:33 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:33.013+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:33.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:33 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:33.013+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:33.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:33 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:33.013+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:33.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:33 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:33.196+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:33.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:33 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:33.196+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:33.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:33 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:33.196+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:33.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:33 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:33.196+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:33.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:33 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:33.446+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:33.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:33 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:33.446+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:33.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:33 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:33.446+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.435816+0000 front 2024-03-19T02:05:03.435838+0000 (oldest deadline 2024-03-19T02:05:28.735708+0000) 2024-03-19T02:05:34.008 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:33 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:33.705+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:34.008 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:33 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:33.705+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:34.008 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:33 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:33.705+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:34.008 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:33 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:33.705+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:34.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:33 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:33.911+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:34.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:33 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:33.911+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:34.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:33 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:33.911+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:34.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:33 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:33.911+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:34.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:34 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:34.007+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:34.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:34 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:34.007+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:34.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:34 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:34.007+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:34.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:34.216+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:34.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:34.216+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:34.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:34.216+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:34.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:34 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:34.216+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:34.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:34.431+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:34.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:34.431+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:34.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:34.431+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.435816+0000 front 2024-03-19T02:05:03.435838+0000 (oldest deadline 2024-03-19T02:05:28.735708+0000) 2024-03-19T02:05:34.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:34.431+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.037493+0000 front 2024-03-19T02:05:12.037520+0000 (oldest deadline 2024-03-19T02:05:34.337696+0000) 2024-03-19T02:05:34.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:34 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:34.664+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:34.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:34 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:34.664+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:34.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:34 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:34.664+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:34.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:34 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:34.664+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:35.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:34.949+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:35.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:34.949+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:35.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:34.949+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:35.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:34.949+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:35.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:34.949+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:35.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:35.000+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:35.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:35.000+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:35.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:35.000+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:35.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:35 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:35.183+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:35.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:35 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:35.183+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:35.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:35 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:35.183+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:35.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:35 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:35.183+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:35.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:35 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:35.467+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:35.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:35 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:35.467+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:35.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:35 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:35.467+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.435816+0000 front 2024-03-19T02:05:03.435838+0000 (oldest deadline 2024-03-19T02:05:28.735708+0000) 2024-03-19T02:05:35.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:35 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:35.467+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.037493+0000 front 2024-03-19T02:05:12.037520+0000 (oldest deadline 2024-03-19T02:05:34.337696+0000) 2024-03-19T02:05:35.982 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:35.707+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:35.982 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:35.707+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:35.982 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:35.707+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:35.982 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:35.707+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:36.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:35 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:35.979+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:36.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:35 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:35.979+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:36.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:35 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:35.979+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:36.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:35 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:35.979+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:36.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:35 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:35.979+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:36.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:35.982+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:36.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:35.982+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:36.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:35.982+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:36.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:35 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:35.982+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:36.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:36.224+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:36.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:36.224+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:36.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:36.224+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:36.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:36 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:36.224+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:36.846 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:36 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:36.491+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.832561+0000 front 2024-03-19T02:04:44.832644+0000 (oldest deadline 2024-03-19T02:05:09.532709+0000) 2024-03-19T02:05:36.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:36 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:36.491+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.433931+0000 front 2024-03-19T02:04:55.433851+0000 (oldest deadline 2024-03-19T02:05:18.933845+0000) 2024-03-19T02:05:36.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:36 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:36.491+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.435816+0000 front 2024-03-19T02:05:03.435838+0000 (oldest deadline 2024-03-19T02:05:28.735708+0000) 2024-03-19T02:05:36.847 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:36 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7[85589]: 2024-03-19T02:05:36.491+0000 7f96bbc48700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.037493+0000 front 2024-03-19T02:05:12.037520+0000 (oldest deadline 2024-03-19T02:05:34.337696+0000) 2024-03-19T02:05:37.003 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:36 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:36.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:37.003 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:36 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:36.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:37.003 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:36 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:36.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:37.003 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:36 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:36.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:37.003 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:36 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:36.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:37.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:36 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:36.981+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:37.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:36 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:36.981+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:37.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:36 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:36.981+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:37.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:36 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:36.981+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:37.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:36 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:36.981+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:37.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:37 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:37.002+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:37.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:37 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:37.002+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:37.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:37 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:37.002+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:37.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:37 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:37.002+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:37.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:37 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:37.203+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:37.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:37 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:37.203+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:37.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:37 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:37.203+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:37.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:37 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:37.203+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:38.008 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:37 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:37.725+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:38.008 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:37 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:37.725+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:38.009 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:37 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:37.725+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:38.009 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:37 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:37.725+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:38.009 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:37 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:37.725+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:38.059 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:37 smithi144 podman[103845]: 2024-03-19 02:05:37.79336775 +0000 UTC m=+5.618088445 container died 69da6b9bf2ef29bda4de2294b5909da6eca12b03402143de3a2be486175d66ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-03-19T02:05:38.059 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.017+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:38.059 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.017+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:38.059 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.017+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:38.059 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.017+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:38.059 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.017+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:38.346 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:38 smithi144 podman[103845]: 2024-03-19 02:05:38.166693923 +0000 UTC m=+5.991414610 container cleanup 69da6b9bf2ef29bda4de2294b5909da6eca12b03402143de3a2be486175d66ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:05:38.346 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:38 smithi144 bash[103845]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7 2024-03-19T02:05:38.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:38.007+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:38.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:38.007+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:38.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:38.007+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:38.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:38.007+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:38.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:38 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:38.209+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:38.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:38 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:38.209+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:38.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:38 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:38.209+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:38.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:38 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:38.209+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:38.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:38 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:38.209+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:38.784 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:38 smithi144 podman[103863]: 2024-03-19 02:05:38.443124279 +0000 UTC m=+0.645043264 container remove 69da6b9bf2ef29bda4de2294b5909da6eca12b03402143de3a2be486175d66ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:05:39.031 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:38.713+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:39.031 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:38.713+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:39.031 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:38.713+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:39.031 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:38.713+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:39.031 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:38 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:38.713+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:39.071 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:38 smithi144 podman[103953]: 2024-03-19 02:05:38.687237796 +0000 UTC m=+0.017670986 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:05:39.071 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:38 smithi144 podman[103953]: 2024-03-19 02:05:38.785376865 +0000 UTC m=+0.115810043 container create dcb147b33a6530ef02970a0538ee40e8ad5b0dddcfbaef47abe5370e8cfd72ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:05:39.072 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.992+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:39.072 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.992+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:39.072 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.992+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:39.072 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.992+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:39.072 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:38 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:38.992+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:39.300 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:39 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:39.030+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:39.300 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:39 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:39.030+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:39.300 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:39 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:39.030+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:39.300 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:39 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:39.030+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:39.346 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:39 smithi144 podman[103953]: 2024-03-19 02:05:39.072063873 +0000 UTC m=+0.402497055 container init dcb147b33a6530ef02970a0538ee40e8ad5b0dddcfbaef47abe5370e8cfd72ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:05:39.347 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:39 smithi144 podman[103953]: 2024-03-19 02:05:39.075967663 +0000 UTC m=+0.406400836 container start dcb147b33a6530ef02970a0538ee40e8ad5b0dddcfbaef47abe5370e8cfd72ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-03-19T02:05:39.347 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:39 smithi144 podman[103953]: 2024-03-19 02:05:39.172798561 +0000 UTC m=+0.503231749 container attach dcb147b33a6530ef02970a0538ee40e8ad5b0dddcfbaef47abe5370e8cfd72ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-19T02:05:39.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:39 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:39.228+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:39.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:39 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:39.228+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:39.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:39 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:39.228+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:39.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:39 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:39.228+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:39.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:39 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:39.228+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:39.668 INFO:journalctl@ceph.osd.7.smithi144.stdout:Mar 19 02:05:39 smithi144 podman[103953]: 2024-03-19 02:05:39.362597454 +0000 UTC m=+0.693030633 container died dcb147b33a6530ef02970a0538ee40e8ad5b0dddcfbaef47abe5370e8cfd72ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-19T02:05:40.008 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.7.service' 2024-03-19T02:05:40.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:05:40.053 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-19T02:05:40.053 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-19T02:05:40.053 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.8 2024-03-19T02:05:40.072 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:39 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:39.737+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:40.072 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:39 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:39.737+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:40.072 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:39 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:39.737+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:40.072 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:39 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:39.737+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:40.072 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:39 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:39.737+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:40.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:40 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:40.006+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:40.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:40 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:40.006+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:40.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:40 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:40.006+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:40.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:40 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:40.006+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:40.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:40 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:40.006+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:40.364 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:40.071+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:40.364 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:40.071+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:40.364 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:40.072+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:40.364 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:40.072+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:40.364 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:40 smithi159 systemd[1]: Stopping Ceph osd.8 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:05:40.364 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:40.363+0000 7f40556b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:05:40.364 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:40.363+0000 7f40556b5700 -1 osd.8 90 *** Got signal Terminated *** 2024-03-19T02:05:40.364 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:40.363+0000 7f40556b5700 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:05:40.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:40 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:40.203+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:40.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:40 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:40.203+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:40.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:40 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:40.203+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:40.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:40 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:40.203+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:40.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:40 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:40.203+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:41.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:40.718+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:41.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:40.718+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:41.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:40.718+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:41.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:40.718+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:41.048 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:40 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:40.718+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:41.351 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:41 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:41.031+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:41.351 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:41 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:41.031+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:41.351 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:41 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:41.031+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:41.351 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:41 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:41.031+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:41.351 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:41 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:41.031+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:41.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:41 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:41.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:41.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:41 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:41.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:41.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:41 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:41.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:41.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:41 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:41.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:41.565 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:41 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:41.185+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:41.565 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:41 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:41.185+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:41.565 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:41 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:41.185+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:41.565 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:41 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:41.185+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:41.565 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:41 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:41.185+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:42.080 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:41 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:41.749+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:42.080 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:41 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:41.749+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:42.080 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:41 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:41.749+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:42.081 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:41 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:41.749+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:42.081 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:41 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:41.749+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:42.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:42 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:42.226+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:42.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:42 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:42.226+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:42.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:42 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:42.226+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:42.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:42 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:42.226+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:42.311 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:42 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:42.226+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:42.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:42 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:42.064+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:42.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:42 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:42.064+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:42.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:42 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:42.064+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:42.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:42 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:42.064+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:42.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:42 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:42.064+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:42.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:42 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:42.080+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:42.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:42 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:42.080+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:42.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:42 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:42.080+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:42.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:42 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:42.080+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:43.050 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:42 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:42.791+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:43.050 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:42 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:42.791+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:43.050 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:42 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:42.791+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:43.050 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:42 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:42.791+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:43.050 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:42 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:42.791+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:43.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:43 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:43.042+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:43.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:43 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:43.042+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:43.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:43 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:43.042+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:43.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:43 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:43.042+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:43.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:43 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:43.042+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:43.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:43 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:43.042+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:43.455 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:43.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:43.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:43.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:43.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:43.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:43.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:43.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:43.456 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:43.048+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:20.928084+0000 front 2024-03-19T02:05:20.928067+0000 (oldest deadline 2024-03-19T02:05:42.628173+0000) 2024-03-19T02:05:43.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:43 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:43.177+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:43.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:43 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:43.177+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:43.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:43 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:43.177+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:43.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:43 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:43.177+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:43.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:43 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:43.177+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:44.205 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:44.084+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:44.205 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:44.084+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:44.205 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:44.084+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:44.206 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:44.084+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:44.206 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:44.084+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:20.928084+0000 front 2024-03-19T02:05:20.928067+0000 (oldest deadline 2024-03-19T02:05:42.628173+0000) 2024-03-19T02:05:44.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:43.836+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:44.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:43.836+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:44.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:43.836+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:44.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:43.836+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:44.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:43 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:43.836+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:44.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:44.038+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:44.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:44.038+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:44.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:44.038+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:44.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:44.038+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:44.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:44.038+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:44.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:44 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:44.038+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:44.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:44 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:44.161+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:44.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:44 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:44.161+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:44.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:44 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:44.161+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:44.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:44 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:44.161+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:44.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:44 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:44.161+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:44.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:44 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:44.161+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:45.205 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:45.054+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.722118+0000 front 2024-03-19T02:04:40.722280+0000 (oldest deadline 2024-03-19T02:05:02.421979+0000) 2024-03-19T02:05:45.206 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:45.054+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.424422+0000 front 2024-03-19T02:04:53.424566+0000 (oldest deadline 2024-03-19T02:05:23.425133+0000) 2024-03-19T02:05:45.206 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:45.054+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.425632+0000 front 2024-03-19T02:05:03.425714+0000 (oldest deadline 2024-03-19T02:05:26.325463+0000) 2024-03-19T02:05:45.206 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:45.054+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.726993+0000 front 2024-03-19T02:05:12.727010+0000 (oldest deadline 2024-03-19T02:05:35.027090+0000) 2024-03-19T02:05:45.206 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8[86028]: 2024-03-19T02:05:45.054+0000 7f4051ca9700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:20.928084+0000 front 2024-03-19T02:05:20.928067+0000 (oldest deadline 2024-03-19T02:05:42.628173+0000) 2024-03-19T02:05:45.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:44.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:45.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:44.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:45.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:44.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:45.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:44.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:45.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:44 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:44.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:45.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:45 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:45.005+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:45.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:45 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:45.005+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:45.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:45 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:45.005+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:45.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:45 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:45.005+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:45.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:45 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:45.005+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:45.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:45 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:45.005+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:45.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:45 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:45.189+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:45.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:45 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:45.189+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:45.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:45 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:45.189+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:45.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:45 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:45.189+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:45.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:45 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:45.189+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:45.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:45 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:45.189+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:46.080 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:45 smithi159 podman[103836]: 2024-03-19 02:05:45.794013836 +0000 UTC m=+5.550017511 container died 22c441205f065441d766c1721cf16ed1e9199b7f696219a9efc29f3ae758c8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-19T02:05:46.081 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:45.930+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:46.081 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:45.930+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:46.081 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:45.930+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:46.081 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:45.930+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:46.081 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:45.930+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:46.081 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:45 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:45.930+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:46.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:46 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:46.045+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:46.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:46 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:46.045+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:46.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:46 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:46.045+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:46.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:46 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:46.045+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:46.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:46 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:46.045+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:46.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:46 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:46.045+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:46.431 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:46 smithi159 podman[103836]: 2024-03-19 02:05:46.172231024 +0000 UTC m=+5.928234669 container cleanup 22c441205f065441d766c1721cf16ed1e9199b7f696219a9efc29f3ae758c8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-19T02:05:46.431 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:46 smithi159 bash[103836]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8 2024-03-19T02:05:46.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:46 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:46.139+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:46.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:46 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:46.139+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:46.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:46 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:46.139+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:46.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:46 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:46.139+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:46.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:46 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:46.139+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:46.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:46 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:46.139+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:46.705 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:46 smithi159 podman[103853]: 2024-03-19 02:05:46.431816231 +0000 UTC m=+0.632156947 container remove 22c441205f065441d766c1721cf16ed1e9199b7f696219a9efc29f3ae758c8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:05:47.043 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:46 smithi159 podman[103943]: 2024-03-19 02:05:46.68828902 +0000 UTC m=+0.017629178 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:05:47.043 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:46 smithi159 podman[103943]: 2024-03-19 02:05:46.824601431 +0000 UTC m=+0.153941568 container create fa7baf076ababb0ea125e342a5cd38971832d65f7be83f6636a591d8e05ed263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-19T02:05:47.043 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:47 smithi159 podman[103943]: 2024-03-19 02:05:47.043715576 +0000 UTC m=+0.373055718 container init fa7baf076ababb0ea125e342a5cd38971832d65f7be83f6636a591d8e05ed263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:05:47.043 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:46 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:46.888+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:47.043 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:46 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:46.888+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:47.044 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:46 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:46.888+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:47.044 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:46 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:46.888+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:47.044 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:46 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:46.888+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:47.044 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:46 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:46.888+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:47.338 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:47 smithi159 podman[103943]: 2024-03-19 02:05:47.046905618 +0000 UTC m=+0.376245764 container start fa7baf076ababb0ea125e342a5cd38971832d65f7be83f6636a591d8e05ed263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-19T02:05:47.338 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:47 smithi159 podman[103943]: 2024-03-19 02:05:47.126900423 +0000 UTC m=+0.456240585 container attach fa7baf076ababb0ea125e342a5cd38971832d65f7be83f6636a591d8e05ed263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-19T02:05:47.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:47 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:47.081+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:47.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:47 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:47.081+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:47.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:47 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:47.081+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:47.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:47 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:47.081+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:47.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:47 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:47.081+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:47.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:47 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:47.081+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:47.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:47 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:47.179+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:47.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:47 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:47.179+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:47.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:47 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:47.179+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:47.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:47 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:47.179+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:47.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:47 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:47.179+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:47.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:47 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:47.179+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:47.638 INFO:journalctl@ceph.osd.8.smithi159.stdout:Mar 19 02:05:47 smithi159 podman[103943]: 2024-03-19 02:05:47.339009797 +0000 UTC m=+0.668349937 container died fa7baf076ababb0ea125e342a5cd38971832d65f7be83f6636a591d8e05ed263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-19T02:05:47.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:47 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:47.838+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:47.955 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:47 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:47.838+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:47.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:47 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:47.838+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:47.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:47 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:47.838+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:47.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:47 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:47.838+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:47.956 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:47 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:47.838+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:48.011 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.8.service' 2024-03-19T02:05:48.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:05:48.094 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-19T02:05:48.094 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-19T02:05:48.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.9 2024-03-19T02:05:48.402 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 systemd[1]: Stopping Ceph osd.9 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:05:48.402 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:48.151+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:48.403 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:48.151+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:48.403 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:48.151+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:48.403 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:48.151+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:48.403 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:48.151+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:48.403 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:48.151+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:48.403 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:48.402+0000 7f235dde7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:05:48.403 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:48.402+0000 7f235dde7700 -1 osd.9 90 *** Got signal Terminated *** 2024-03-19T02:05:48.403 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:48 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:48.402+0000 7f235dde7700 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:05:48.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:48 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:48.123+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:48.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:48 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:48.123+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:48.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:48 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:48.123+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:48.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:48 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:48.123+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:48.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:48 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:48.123+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:48.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:48 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:48.123+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:49.185 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:48 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:48.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:49.186 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:48 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:48.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:49.186 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:48 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:48.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:49.186 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:48 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:48.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:49.186 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:48 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:48.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:49.186 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:48 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:48.882+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:49.425 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:49 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:49.102+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:49.425 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:49 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:49.102+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:49.425 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:49 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:49.102+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:49.425 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:49 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:49.102+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:49.425 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:49 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:49.102+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:49.426 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:49 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:49.102+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:49.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:49 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:49.154+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:49.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:49 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:49.154+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:49.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:49 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:49.154+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:49.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:49 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:49.154+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:49.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:49 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:49.154+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:49.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:49 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:49.154+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:50.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:49 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:49.919+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:50.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:49 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:49.919+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:50.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:49 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:49.919+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:50.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:49 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:49.919+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:50.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:49 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:49.919+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:50.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:49 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:49.919+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:50.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:50 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:50.086+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:50.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:50 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:50.086+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:50.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:50 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:50.086+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:50.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:50 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:50.086+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:50.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:50 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:50.086+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:50.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:50 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:50.086+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:50.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:50 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:50.166+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:50.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:50 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:50.166+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:50.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:50 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:50.166+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:50.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:50 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:50.166+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:50.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:50 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:50.166+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:50.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:50 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:50.166+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:51.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:50 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:50.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:51.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:50 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:50.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:51.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:50 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:50.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:51.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:50 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:50.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:51.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:50 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:50.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:51.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:50 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:50.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:51.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:51 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:51.061+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:51.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:51 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:51.061+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:51.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:51 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:51.061+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:51.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:51 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:51.061+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:51.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:51 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:51.061+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:51.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:51 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:51.061+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:51.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:51 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:51.061+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:05:51.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:51 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:51.134+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:51.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:51 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:51.134+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:51.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:51 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:51.134+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:51.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:51 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:51.134+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:51.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:51 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:51.134+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:51.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:51 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:51.134+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:52.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:51 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:51.828+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:52.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:51 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:51.828+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:52.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:51 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:51.828+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:52.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:51 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:51.828+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:52.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:51 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:51.828+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:52.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:51 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:51.828+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:52.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:52 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:52.079+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:52.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:52 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:52.079+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:52.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:52 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:52.079+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:52.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:52 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:52.079+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:52.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:52 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:52.079+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:52.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:52 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:52.079+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:52.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:52 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:52.079+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:05:52.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:52 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:52.108+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:52.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:52 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:52.108+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:52.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:52 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:52.108+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:52.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:52 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:52.108+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:52.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:52 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:52.108+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:52.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:52 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:52.108+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:53.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:52 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:52.862+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:53.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:52 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:52.862+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:53.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:52 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:52.862+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:53.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:52 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:52.862+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:53.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:52 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:52.862+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:53.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:52 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:52.862+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:53.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:53 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:53.032+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:53.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:53 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:53.032+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:53.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:53 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:53.032+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:53.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:53 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:53.032+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:53.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:53 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:53.032+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:53.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:53 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:53.032+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:53.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:53 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:53.032+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:05:53.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:53.060+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:40.048543+0000 front 2024-03-19T02:04:40.048570+0000 (oldest deadline 2024-03-19T02:05:02.948674+0000) 2024-03-19T02:05:53.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:53.060+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.150493+0000 front 2024-03-19T02:04:48.150356+0000 (oldest deadline 2024-03-19T02:05:12.150884+0000) 2024-03-19T02:05:53.560 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:53.060+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:55.052004+0000 front 2024-03-19T02:04:55.052141+0000 (oldest deadline 2024-03-19T02:05:18.551990+0000) 2024-03-19T02:05:53.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:53.060+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:00.753548+0000 front 2024-03-19T02:05:00.753561+0000 (oldest deadline 2024-03-19T02:05:26.653592+0000) 2024-03-19T02:05:53.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:53.060+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:13.055248+0000 front 2024-03-19T02:05:13.055231+0000 (oldest deadline 2024-03-19T02:05:37.754984+0000) 2024-03-19T02:05:53.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:53.060+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.256114+0000 front 2024-03-19T02:05:18.256240+0000 (oldest deadline 2024-03-19T02:05:43.556078+0000) 2024-03-19T02:05:53.561 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:53 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9[97739]: 2024-03-19T02:05:53.060+0000 7f235a3db700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:28.157497+0000 front 2024-03-19T02:05:28.157461+0000 (oldest deadline 2024-03-19T02:05:52.857575+0000) 2024-03-19T02:05:53.976 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:53 smithi092 podman[122539]: 2024-03-19 02:05:53.719562822 +0000 UTC m=+5.434424075 container died faf050e74e25a67702c557fe607ea8b821960a45d22b35554c23d697504943cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:05:54.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:53.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:54.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:53.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:54.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:53.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:54.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:53.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:54.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:53.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:54.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:53.876+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:54.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:54 smithi092 podman[122539]: 2024-03-19 02:05:54.075907024 +0000 UTC m=+5.790768273 container cleanup faf050e74e25a67702c557fe607ea8b821960a45d22b35554c23d697504943cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:05:54.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:54 smithi092 bash[122539]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9 2024-03-19T02:05:54.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:54 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:54.069+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:54.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:54 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:54.069+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:54.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:54 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:54.069+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:54.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:54 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:54.069+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:54.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:54 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:54.069+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:54.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:54 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:54.069+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:54.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:54 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:54.069+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:05:54.735 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:54 smithi092 podman[122555]: 2024-03-19 02:05:54.385423211 +0000 UTC m=+0.661448720 container remove faf050e74e25a67702c557fe607ea8b821960a45d22b35554c23d697504943cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:05:55.046 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:54 smithi092 podman[122646]: 2024-03-19 02:05:54.73609655 +0000 UTC m=+0.114091765 container create efb58c61068626c042c0d73e6db78b178694a8eea3b6e7d72fd81ffdfdb479ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-19T02:05:55.046 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:54 smithi092 podman[122646]: 2024-03-19 02:05:54.640304692 +0000 UTC m=+0.018299926 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:05:55.046 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:54 smithi092 podman[122646]: 2024-03-19 02:05:54.971762404 +0000 UTC m=+0.349757620 container init efb58c61068626c042c0d73e6db78b178694a8eea3b6e7d72fd81ffdfdb479ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:05:55.046 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:54 smithi092 podman[122646]: 2024-03-19 02:05:54.974762536 +0000 UTC m=+0.352757761 container start efb58c61068626c042c0d73e6db78b178694a8eea3b6e7d72fd81ffdfdb479ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9-deactivate, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-19T02:05:55.046 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:55 smithi092 podman[122646]: 2024-03-19 02:05:55.046733604 +0000 UTC m=+0.424728829 container attach efb58c61068626c042c0d73e6db78b178694a8eea3b6e7d72fd81ffdfdb479ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:05:55.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:54.848+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:55.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:54.848+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:55.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:54.848+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:55.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:54.848+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:55.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:54.848+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:55.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:54 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:54.848+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:55.310 INFO:journalctl@ceph.osd.9.smithi092.stdout:Mar 19 02:05:55 smithi092 podman[122646]: 2024-03-19 02:05:55.268327467 +0000 UTC m=+0.646322684 container died efb58c61068626c042c0d73e6db78b178694a8eea3b6e7d72fd81ffdfdb479ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:05:55.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:55 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:55.118+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:55.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:55 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:55.118+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:55.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:55 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:55.118+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:55.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:55 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:55.118+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:55.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:55 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:55.118+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:55.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:55 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:55.118+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:55.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:55 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:55.118+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:05:55.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.9.service' 2024-03-19T02:05:55.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:05:55.865 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-19T02:05:55.865 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-19T02:05:55.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.10 2024-03-19T02:05:56.171 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:55 smithi144 systemd[1]: Stopping Ceph osd.10 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:05:56.171 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.114+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:56.171 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.114+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:56.171 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.114+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:56.171 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.114+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:56.171 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.114+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:56.171 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.114+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:56.171 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.114+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:05:56.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:55.814+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:56.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:55.814+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:56.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:55.814+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:56.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:55.814+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:56.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:55.814+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:56.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:55 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:55.814+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:56.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.170+0000 7fd6a7941700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:05:56.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.170+0000 7fd6a7941700 -1 osd.10 90 *** Got signal Terminated *** 2024-03-19T02:05:56.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:56 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:56.170+0000 7fd6a7941700 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:05:57.107 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:56 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:56.829+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:57.107 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:56 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:56.829+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:57.107 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:56 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:56.829+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:57.107 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:56 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:56.829+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:57.107 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:56 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:56.829+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:57.107 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:56 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:56.829+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:57.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:57.149+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:57.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:57.149+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:57.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:57.149+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:57.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:57.149+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:57.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:57.149+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:57.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:57.149+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:57.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:57 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:57.149+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:05:58.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:57.823+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:58.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:57.823+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:58.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:57.823+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:58.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:57.823+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:58.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:57.823+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:58.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:57 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:57.823+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:58.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:58.140+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:58.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:58.140+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:58.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:58.140+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:58.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:58.140+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:58.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:58.140+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:58.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:58.140+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:58.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:58 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:58.140+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:05:59.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:58.834+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:05:59.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:58.834+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:05:59.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:58.834+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:05:59.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:58.834+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:05:59.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:58.834+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:05:59.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:58.834+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:05:59.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:58.834+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:05:59.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:59.099+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:05:59.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:59.099+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:05:59.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:59.099+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:05:59.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:59.099+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:05:59.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:59.099+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:05:59.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:59.099+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:05:59.597 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:05:59 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:05:59.099+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:06:00.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:59.816+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:00.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:59.816+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:00.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:59.816+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:00.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:59.816+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:00.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:59.816+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:00.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:59.816+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:00.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:05:59 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:05:59.816+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:00.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:00.062+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:06:00.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:00.062+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:06:00.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:00.062+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:06:00.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:00.062+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:06:00.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:00.062+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:06:00.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:00.062+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:06:00.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:00 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:00.062+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:06:01.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:00.797+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:01.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:00.797+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:01.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:00.797+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:01.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:00.797+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:01.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:00.797+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:01.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:00.797+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:01.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:00 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:00.797+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:01.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:01 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:01.065+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.929667+0000 front 2024-03-19T02:04:39.929709+0000 (oldest deadline 2024-03-19T02:05:03.429626+0000) 2024-03-19T02:06:01.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:01 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:01.065+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:48.130819+0000 front 2024-03-19T02:04:48.130722+0000 (oldest deadline 2024-03-19T02:05:10.331595+0000) 2024-03-19T02:06:01.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:01 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:01.065+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:57.233847+0000 front 2024-03-19T02:04:57.233773+0000 (oldest deadline 2024-03-19T02:05:17.733707+0000) 2024-03-19T02:06:01.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:01 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:01.065+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:03.535295+0000 front 2024-03-19T02:05:03.535237+0000 (oldest deadline 2024-03-19T02:05:25.835065+0000) 2024-03-19T02:06:01.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:01 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:01.065+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:09.935976+0000 front 2024-03-19T02:05:09.935933+0000 (oldest deadline 2024-03-19T02:05:34.636027+0000) 2024-03-19T02:06:01.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:01 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:01.065+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:18.037884+0000 front 2024-03-19T02:05:18.037675+0000 (oldest deadline 2024-03-19T02:05:42.137553+0000) 2024-03-19T02:06:01.347 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:01 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10[92503]: 2024-03-19T02:06:01.065+0000 7fd6a3f35700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.92:6822 osd.6 since back 2024-03-19T02:05:27.740057+0000 front 2024-03-19T02:05:27.740069+0000 (oldest deadline 2024-03-19T02:05:50.540459+0000) 2024-03-19T02:06:01.905 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:01 smithi144 podman[104058]: 2024-03-19 02:06:01.600598651 +0000 UTC m=+5.547505102 container died c86852528659ba2ff95212b2796c9dc3363521509be32b04b3a8170e402c238f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-19T02:06:02.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:01 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:01.783+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:02.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:01 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:01.783+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:02.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:01 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:01.783+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:02.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:01 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:01.783+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:02.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:01 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:01.783+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:02.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:01 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:01.783+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:02.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:01 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:01.783+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:02.254 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:02 smithi144 podman[104058]: 2024-03-19 02:06:02.004269339 +0000 UTC m=+5.951175789 container cleanup c86852528659ba2ff95212b2796c9dc3363521509be32b04b3a8170e402c238f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:06:02.254 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:02 smithi144 bash[104058]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10 2024-03-19T02:06:02.596 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:02 smithi144 podman[104076]: 2024-03-19 02:06:02.255131116 +0000 UTC m=+0.649622334 container remove c86852528659ba2ff95212b2796c9dc3363521509be32b04b3a8170e402c238f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:06:02.900 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:02 smithi144 podman[104164]: 2024-03-19 02:06:02.606110302 +0000 UTC m=+0.113610044 container create 37fcb9afcd2e968228e98d1453f1a0f0fc60dd44b32e87307b369ff0854f8358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:06:02.900 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:02 smithi144 podman[104164]: 2024-03-19 02:06:02.510055825 +0000 UTC m=+0.017555595 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:06:02.900 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:02 smithi144 podman[104164]: 2024-03-19 02:06:02.791670214 +0000 UTC m=+0.299169966 container init 37fcb9afcd2e968228e98d1453f1a0f0fc60dd44b32e87307b369ff0854f8358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-19T02:06:02.900 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:02 smithi144 podman[104164]: 2024-03-19 02:06:02.795506297 +0000 UTC m=+0.303006041 container start 37fcb9afcd2e968228e98d1453f1a0f0fc60dd44b32e87307b369ff0854f8358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-19T02:06:02.900 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:02 smithi144 podman[104164]: 2024-03-19 02:06:02.90016685 +0000 UTC m=+0.407666608 container attach 37fcb9afcd2e968228e98d1453f1a0f0fc60dd44b32e87307b369ff0854f8358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-19T02:06:03.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:02 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:02.777+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:03.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:02 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:02.777+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:03.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:02 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:02.777+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:03.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:02 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:02.777+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:03.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:02 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:02.777+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:03.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:02 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:02.777+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:03.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:02 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:02.777+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:03.346 INFO:journalctl@ceph.osd.10.smithi144.stdout:Mar 19 02:06:03 smithi144 podman[104164]: 2024-03-19 02:06:03.085760058 +0000 UTC m=+0.593259802 container died 37fcb9afcd2e968228e98d1453f1a0f0fc60dd44b32e87307b369ff0854f8358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T02:06:03.591 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.10.service' 2024-03-19T02:06:03.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:03.631 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-19T02:06:03.631 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-19T02:06:03.631 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.11 2024-03-19T02:06:03.949 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 systemd[1]: Stopping Ceph osd.11 for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:03.949 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:03.949 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:03.950 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:03.950 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:03.950 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:03.950 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:03.950 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.755+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:04.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.949+0000 7fc9d70b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:04.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.949+0000 7fc9d70b2700 -1 osd.11 90 *** Got signal Terminated *** 2024-03-19T02:06:04.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:03 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:03.949+0000 7fc9d70b2700 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-19T02:06:05.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:04 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:04.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:05.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:04 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:04.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:05.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:04 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:04.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:05.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:04 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:04.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:05.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:04 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:04.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:05.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:04 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:04.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:05.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:04 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:04.745+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:06.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:05.786+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:06.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:05.786+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:06.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:05.786+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:06.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:05.786+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:06.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:05.786+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:06.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:05.786+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:06.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:05 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:05.786+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:07.155 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:06.752+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:07.155 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:06.752+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:07.155 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:06.752+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:07.155 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:06.752+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:07.155 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:06.752+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:07.156 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:06.752+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:07.156 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:06.752+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:07.156 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:06 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:06.752+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6820 osd.8 since back 2024-03-19T02:05:42.322828+0000 front 2024-03-19T02:05:42.322872+0000 (oldest deadline 2024-03-19T02:06:05.822929+0000) 2024-03-19T02:06:08.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:07.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:08.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:07.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:08.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:07.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:08.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:07.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:08.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:07.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:08.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:07.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:08.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:07.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:08.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:07 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:07.710+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6820 osd.8 since back 2024-03-19T02:05:42.322828+0000 front 2024-03-19T02:05:42.322872+0000 (oldest deadline 2024-03-19T02:06:05.822929+0000) 2024-03-19T02:06:09.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:08.723+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6806 osd.0 since back 2024-03-19T02:04:39.513396+0000 front 2024-03-19T02:04:39.513189+0000 (oldest deadline 2024-03-19T02:05:04.813397+0000) 2024-03-19T02:06:09.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:08.723+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6804 osd.1 since back 2024-03-19T02:04:44.814165+0000 front 2024-03-19T02:04:44.814010+0000 (oldest deadline 2024-03-19T02:05:10.114100+0000) 2024-03-19T02:06:09.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:08.723+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-03-19T02:04:53.615064+0000 front 2024-03-19T02:04:53.615212+0000 (oldest deadline 2024-03-19T02:05:18.315122+0000) 2024-03-19T02:06:09.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:08.723+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.92:6814 osd.3 since back 2024-03-19T02:05:05.217242+0000 front 2024-03-19T02:05:05.217401+0000 (oldest deadline 2024-03-19T02:05:28.717165+0000) 2024-03-19T02:06:09.206 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:08.723+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6812 osd.4 since back 2024-03-19T02:05:12.218156+0000 front 2024-03-19T02:05:12.218240+0000 (oldest deadline 2024-03-19T02:05:36.317995+0000) 2024-03-19T02:06:09.207 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:08.723+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-03-19T02:05:19.719234+0000 front 2024-03-19T02:05:19.719214+0000 (oldest deadline 2024-03-19T02:05:45.019190+0000) 2024-03-19T02:06:09.207 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:08.723+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.144:6820 osd.7 since back 2024-03-19T02:05:35.322150+0000 front 2024-03-19T02:05:35.322204+0000 (oldest deadline 2024-03-19T02:05:58.221959+0000) 2024-03-19T02:06:09.207 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:08 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11[92952]: 2024-03-19T02:06:08.723+0000 7fc9d36a6700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.159:6820 osd.8 since back 2024-03-19T02:05:42.322828+0000 front 2024-03-19T02:05:42.322872+0000 (oldest deadline 2024-03-19T02:06:05.822929+0000) 2024-03-19T02:06:09.774 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:09 smithi159 podman[104061]: 2024-03-19 02:06:09.491554787 +0000 UTC m=+5.665633809 container died f93f0f685ef29087981c4db3f61ef01ef810f330016727b592df49e28a475ef9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-19T02:06:10.135 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:09 smithi159 podman[104061]: 2024-03-19 02:06:09.874206075 +0000 UTC m=+6.048285094 container cleanup f93f0f685ef29087981c4db3f61ef01ef810f330016727b592df49e28a475ef9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-19T02:06:10.135 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:09 smithi159 bash[104061]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11 2024-03-19T02:06:10.455 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:10 smithi159 podman[104079]: 2024-03-19 02:06:10.135861698 +0000 UTC m=+0.634941822 container remove f93f0f685ef29087981c4db3f61ef01ef810f330016727b592df49e28a475ef9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-19T02:06:10.763 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:10 smithi159 podman[104169]: 2024-03-19 02:06:10.478563116 +0000 UTC m=+0.103342433 container create ed4a40027466719ef6b64006140c4995e236f60d2c0609609cca351dad1bc61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-19T02:06:10.763 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:10 smithi159 podman[104169]: 2024-03-19 02:06:10.393539672 +0000 UTC m=+0.018319004 image pull 2312c8e42d0069a2dbb2364bddb00264db4b800e7f18a7e7e4eb8f3ecab2ff1c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639 2024-03-19T02:06:10.763 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:10 smithi159 podman[104169]: 2024-03-19 02:06:10.68056498 +0000 UTC m=+0.305344310 container init ed4a40027466719ef6b64006140c4995e236f60d2c0609609cca351dad1bc61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:06:10.763 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:10 smithi159 podman[104169]: 2024-03-19 02:06:10.683947438 +0000 UTC m=+0.308726756 container start ed4a40027466719ef6b64006140c4995e236f60d2c0609609cca351dad1bc61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-19T02:06:10.763 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:10 smithi159 podman[104169]: 2024-03-19 02:06:10.763971324 +0000 UTC m=+0.388750662 container attach ed4a40027466719ef6b64006140c4995e236f60d2c0609609cca351dad1bc61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-19T02:06:11.205 INFO:journalctl@ceph.osd.11.smithi159.stdout:Mar 19 02:06:10 smithi159 podman[104169]: 2024-03-19 02:06:10.972005233 +0000 UTC m=+0.596784555 container died ed4a40027466719ef6b64006140c4995e236f60d2c0609609cca351dad1bc61d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-19T02:06:11.564 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@osd.11.service' 2024-03-19T02:06:11.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:11.605 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-19T02:06:11.605 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-19T02:06:11.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.a 2024-03-19T02:06:11.893 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:06:11 smithi092 systemd[1]: Stopping Ceph mds.a for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:12.310 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:06:11 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a[104547]: 2024-03-19T02:06:11.892+0000 7f8410d40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:12.311 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:06:11 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a[104547]: 2024-03-19T02:06:11.892+0000 7f8410d40700 -1 mds.a *** got signal Terminated *** 2024-03-19T02:06:16.159 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:06:15 smithi092 podman[122774]: 2024-03-19 02:06:15.899984807 +0000 UTC m=+4.098921783 container died 93b782d6d8b8373ea2e3df52471eef7dc082298bb4a39e5c7d4c64d25686ea64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307) 2024-03-19T02:06:16.434 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:06:16 smithi092 podman[122774]: 2024-03-19 02:06:16.233543484 +0000 UTC m=+4.432480458 container cleanup 93b782d6d8b8373ea2e3df52471eef7dc082298bb4a39e5c7d4c64d25686ea64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-19T02:06:16.434 INFO:journalctl@ceph.mds.a.smithi092.stdout:Mar 19 02:06:16 smithi092 bash[122774]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-a 2024-03-19T02:06:16.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.a.service' 2024-03-19T02:06:16.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:16.693 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-19T02:06:16.693 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-19T02:06:16.693 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.d 2024-03-19T02:06:17.012 INFO:journalctl@ceph.mds.d.smithi092.stdout:Mar 19 02:06:16 smithi092 systemd[1]: Stopping Ceph mds.d for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:17.310 INFO:journalctl@ceph.mds.d.smithi092.stdout:Mar 19 02:06:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-d[103576]: 2024-03-19T02:06:17.011+0000 7f9fa5e2c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:17.310 INFO:journalctl@ceph.mds.d.smithi092.stdout:Mar 19 02:06:17 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-d[103576]: 2024-03-19T02:06:17.011+0000 7f9fa5e2c700 -1 mds.d *** got signal Terminated *** 2024-03-19T02:06:21.310 INFO:journalctl@ceph.mds.d.smithi092.stdout:Mar 19 02:06:21 smithi092 podman[122895]: 2024-03-19 02:06:21.018337374 +0000 UTC m=+4.109565869 container died 1e4567c99fe6830dd6eeafcb91f0c81740d7a4f7b4a0785ce1f9d76796a54323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-19T02:06:21.646 INFO:journalctl@ceph.mds.d.smithi092.stdout:Mar 19 02:06:21 smithi092 podman[122895]: 2024-03-19 02:06:21.403767567 +0000 UTC m=+4.494996061 container cleanup 1e4567c99fe6830dd6eeafcb91f0c81740d7a4f7b4a0785ce1f9d76796a54323 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD) 2024-03-19T02:06:21.646 INFO:journalctl@ceph.mds.d.smithi092.stdout:Mar 19 02:06:21 smithi092 bash[122895]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-d 2024-03-19T02:06:21.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.d.service' 2024-03-19T02:06:21.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:21.816 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-19T02:06:21.816 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-19T02:06:21.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.g 2024-03-19T02:06:22.310 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:06:21 smithi092 systemd[1]: Stopping Ceph mds.g for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:22.310 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:06:22 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g[104187]: 2024-03-19T02:06:22.139+0000 7f2b3ee49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:22.310 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:06:22 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g[104187]: 2024-03-19T02:06:22.140+0000 7f2b3ee49700 -1 mds.g *** got signal Terminated *** 2024-03-19T02:06:25.049 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:06:24 smithi092 podman[123016]: 2024-03-19 02:06:24.777721657 +0000 UTC m=+2.745076613 container died 7136f2cd665d249609365725027a761d77af6cba97b9811afde51e76f90527fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-19T02:06:25.310 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:06:25 smithi092 podman[123016]: 2024-03-19 02:06:25.149130603 +0000 UTC m=+3.116485558 container cleanup 7136f2cd665d249609365725027a761d77af6cba97b9811afde51e76f90527fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249) 2024-03-19T02:06:25.310 INFO:journalctl@ceph.mds.g.smithi092.stdout:Mar 19 02:06:25 smithi092 bash[123016]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-g 2024-03-19T02:06:25.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.g.service' 2024-03-19T02:06:25.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:25.600 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-19T02:06:25.600 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-19T02:06:25.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.j 2024-03-19T02:06:26.060 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 02:06:25 smithi092 systemd[1]: Stopping Ceph mds.j for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:26.060 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 02:06:25 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-j[103232]: 2024-03-19T02:06:25.902+0000 7febf749e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:26.061 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 02:06:25 smithi092 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-j[103232]: 2024-03-19T02:06:25.902+0000 7febf749e700 -1 mds.j *** got signal Terminated *** 2024-03-19T02:06:30.293 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 02:06:29 smithi092 podman[123138]: 2024-03-19 02:06:29.908559757 +0000 UTC m=+4.089484977 container died 91ba3f745ff1b0b69bd493d6c06fef9ba370e4b9f0a9f93330a7542b39e6c5fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:06:30.560 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 02:06:30 smithi092 podman[123138]: 2024-03-19 02:06:30.385618053 +0000 UTC m=+4.566543281 container cleanup 91ba3f745ff1b0b69bd493d6c06fef9ba370e4b9f0a9f93330a7542b39e6c5fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-j, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-19T02:06:30.560 INFO:journalctl@ceph.mds.j.smithi092.stdout:Mar 19 02:06:30 smithi092 bash[123138]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-j 2024-03-19T02:06:30.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.j.service' 2024-03-19T02:06:30.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:30.859 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-19T02:06:30.859 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-19T02:06:30.859 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.b 2024-03-19T02:06:31.263 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:30 smithi144 systemd[1]: Stopping Ceph mds.b for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:31.546 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b[97876]: 2024-03-19T02:06:31.262+0000 7fbc73c2d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:31.546 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:31 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b[97876]: 2024-03-19T02:06:31.262+0000 7fbc73c2d700 -1 mds.b *** got signal Terminated *** 2024-03-19T02:06:33.095 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:32 smithi144 podman[104269]: 2024-03-19 02:06:32.790709831 +0000 UTC m=+1.742721571 container died 7fdf2e218f60fce455691b1296733080b9c4d597be00fd52329e271212de798d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-19T02:06:33.346 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:33 smithi144 podman[104269]: 2024-03-19 02:06:33.187863085 +0000 UTC m=+2.139874823 container cleanup 7fdf2e218f60fce455691b1296733080b9c4d597be00fd52329e271212de798d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True) 2024-03-19T02:06:33.797 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:33 smithi144 bash[104269]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b 2024-03-19T02:06:33.798 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.b.service' 2024-03-19T02:06:33.823 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:33 smithi144 podman[104295]: 2024-03-19 02:06:33.405745547 +0000 UTC m=+0.617750205 container remove 7fdf2e218f60fce455691b1296733080b9c4d597be00fd52329e271212de798d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-19T02:06:33.823 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:33 smithi144 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.b.service: Deactivated successfully. 2024-03-19T02:06:33.823 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:33 smithi144 systemd[1]: Stopped Ceph mds.b for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T02:06:33.823 INFO:journalctl@ceph.mds.b.smithi144.stdout:Mar 19 02:06:33 smithi144 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.b.service: Consumed 3min 59.261s CPU time. 2024-03-19T02:06:33.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:33.836 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-19T02:06:33.836 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-19T02:06:33.836 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.e 2024-03-19T02:06:34.096 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:06:33 smithi144 systemd[1]: Stopping Ceph mds.e for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:34.596 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:06:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e[97263]: 2024-03-19T02:06:34.174+0000 7f60c2167700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:34.597 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:06:34 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e[97263]: 2024-03-19T02:06:34.174+0000 7f60c2167700 -1 mds.e *** got signal Terminated *** 2024-03-19T02:06:37.067 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:06:36 smithi144 podman[104404]: 2024-03-19 02:06:36.797599905 +0000 UTC m=+2.748776987 container died 05838348eb27c8d3e2fadbaddca5ee25d8b2742fe56de6ae79ff73d4ec9c31c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-03-19T02:06:37.346 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:06:37 smithi144 podman[104404]: 2024-03-19 02:06:37.167597541 +0000 UTC m=+3.118774613 container cleanup 05838348eb27c8d3e2fadbaddca5ee25d8b2742fe56de6ae79ff73d4ec9c31c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:06:37.346 INFO:journalctl@ceph.mds.e.smithi144.stdout:Mar 19 02:06:37 smithi144 bash[104404]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-e 2024-03-19T02:06:37.583 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.e.service' 2024-03-19T02:06:37.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:37.618 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-19T02:06:37.618 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-19T02:06:37.618 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.h 2024-03-19T02:06:38.096 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:06:37 smithi144 systemd[1]: Stopping Ceph mds.h for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:38.097 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:06:37 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h[97570]: 2024-03-19T02:06:37.945+0000 7f966f5de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:38.097 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:06:37 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h[97570]: 2024-03-19T02:06:37.945+0000 7f966f5de700 -1 mds.h *** got signal Terminated *** 2024-03-19T02:06:41.096 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:06:40 smithi144 podman[104525]: 2024-03-19 02:06:40.78920779 +0000 UTC m=+2.957360156 container died b768b4be8e7b67e47959ab19205f16e252a6e8e9390cd6e9046181371d34ed99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-19T02:06:41.505 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:06:41 smithi144 podman[104525]: 2024-03-19 02:06:41.304606823 +0000 UTC m=+3.472759180 container cleanup b768b4be8e7b67e47959ab19205f16e252a6e8e9390cd6e9046181371d34ed99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-19T02:06:41.505 INFO:journalctl@ceph.mds.h.smithi144.stdout:Mar 19 02:06:41 smithi144 bash[104525]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-h 2024-03-19T02:06:41.713 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.h.service' 2024-03-19T02:06:41.748 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:41.748 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-19T02:06:41.748 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-19T02:06:41.748 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.k 2024-03-19T02:06:42.058 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:06:41 smithi144 systemd[1]: Stopping Ceph mds.k for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:42.346 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:06:42 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k[96942]: 2024-03-19T02:06:42.057+0000 7f305f2f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:42.347 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:06:42 smithi144 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k[96942]: 2024-03-19T02:06:42.057+0000 7f305f2f5700 -1 mds.k *** got signal Terminated *** 2024-03-19T02:06:45.096 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:06:44 smithi144 podman[104647]: 2024-03-19 02:06:44.792717841 +0000 UTC m=+2.836377555 container died e735324ea198e9bbc53f2178329f7ddc068ae07bab04d458ebf050de783842e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-19T02:06:45.596 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:06:45 smithi144 podman[104647]: 2024-03-19 02:06:45.3916233 +0000 UTC m=+3.435282972 container cleanup e735324ea198e9bbc53f2178329f7ddc068ae07bab04d458ebf050de783842e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-19T02:06:45.596 INFO:journalctl@ceph.mds.k.smithi144.stdout:Mar 19 02:06:45 smithi144 bash[104647]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-k 2024-03-19T02:06:45.808 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.k.service' 2024-03-19T02:06:45.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:45.842 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-19T02:06:45.842 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-19T02:06:45.842 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.c 2024-03-19T02:06:46.191 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:06:45 smithi159 systemd[1]: Stopping Ceph mds.c for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:46.455 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:06:46 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c[98418]: 2024-03-19T02:06:46.191+0000 7ff4aa357700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:46.455 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:06:46 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c[98418]: 2024-03-19T02:06:46.191+0000 7ff4aa357700 -1 mds.c *** got signal Terminated *** 2024-03-19T02:06:49.066 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:06:48 smithi159 podman[104274]: 2024-03-19 02:06:48.793954511 +0000 UTC m=+2.759684748 container died 75c7a3f9696f9855e08a807644356e4c8e7139cb9dca7d6f18823c0311ee690d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-19T02:06:49.425 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:06:49 smithi159 podman[104274]: 2024-03-19 02:06:49.166112322 +0000 UTC m=+3.131842557 container cleanup 75c7a3f9696f9855e08a807644356e4c8e7139cb9dca7d6f18823c0311ee690d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-19T02:06:49.425 INFO:journalctl@ceph.mds.c.smithi159.stdout:Mar 19 02:06:49 smithi159 bash[104274]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-c 2024-03-19T02:06:49.586 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.c.service' 2024-03-19T02:06:49.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:49.625 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-19T02:06:49.626 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-19T02:06:49.626 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.f 2024-03-19T02:06:49.945 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:49 smithi159 systemd[1]: Stopping Ceph mds.f for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:50.205 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:49 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f[98119]: 2024-03-19T02:06:49.944+0000 7f1f6a7b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:50.205 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:49 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f[98119]: 2024-03-19T02:06:49.944+0000 7f1f6a7b1700 -1 mds.f *** got signal Terminated *** 2024-03-19T02:06:53.045 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:52 smithi159 podman[104406]: 2024-03-19 02:06:52.793116185 +0000 UTC m=+2.949995559 container died dc3a13d7a5c014ffbc06fc7623b5b7c7d4a6607ddf30a411df34d86fdbb3e162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-19T02:06:53.328 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:53 smithi159 podman[104406]: 2024-03-19 02:06:53.128205592 +0000 UTC m=+3.285084979 container cleanup dc3a13d7a5c014ffbc06fc7623b5b7c7d4a6607ddf30a411df34d86fdbb3e162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-19T02:06:53.329 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:53 smithi159 bash[104406]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f 2024-03-19T02:06:53.555 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.f.service' 2024-03-19T02:06:53.579 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:53 smithi159 podman[104420]: 2024-03-19 02:06:53.329459445 +0000 UTC m=+0.534987625 container remove dc3a13d7a5c014ffbc06fc7623b5b7c7d4a6607ddf30a411df34d86fdbb3e162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-19T02:06:53.579 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:53 smithi159 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.f.service: Deactivated successfully. 2024-03-19T02:06:53.579 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:53 smithi159 systemd[1]: Stopped Ceph mds.f for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T02:06:53.579 INFO:journalctl@ceph.mds.f.smithi159.stdout:Mar 19 02:06:53 smithi159 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.f.service: Consumed 4min 18.696s CPU time. 2024-03-19T02:06:53.591 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:53.591 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-19T02:06:53.592 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-19T02:06:53.592 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.i 2024-03-19T02:06:53.906 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:53 smithi159 systemd[1]: Stopping Ceph mds.i for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:54.205 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: 2024-03-19T02:06:53.905+0000 7f195e97b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:54.205 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:53 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i[98726]: 2024-03-19T02:06:53.906+0000 7f195e97b700 -1 mds.i *** got signal Terminated *** 2024-03-19T02:06:57.115 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:56 smithi159 podman[104528]: 2024-03-19 02:06:56.786142183 +0000 UTC m=+2.979980271 container died 37fa8cc2c9bff735f7a2c71a7c4eca232fdb7ce37efa309b5b58d97d81b213bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-19T02:06:57.440 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:57 smithi159 podman[104528]: 2024-03-19 02:06:57.206626441 +0000 UTC m=+3.400464526 container cleanup 37fa8cc2c9bff735f7a2c71a7c4eca232fdb7ce37efa309b5b58d97d81b213bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-19T02:06:57.441 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:57 smithi159 bash[104528]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i 2024-03-19T02:06:57.677 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.i.service' 2024-03-19T02:06:57.700 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:57 smithi159 podman[104542]: 2024-03-19 02:06:57.441227195 +0000 UTC m=+0.650281571 container remove 37fa8cc2c9bff735f7a2c71a7c4eca232fdb7ce37efa309b5b58d97d81b213bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-19T02:06:57.701 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:57 smithi159 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.i.service: Deactivated successfully. 2024-03-19T02:06:57.701 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:57 smithi159 systemd[1]: Stopped Ceph mds.i for d1e857f4-e591-11ee-95c9-87774f69a715. 2024-03-19T02:06:57.701 INFO:journalctl@ceph.mds.i.smithi159.stdout:Mar 19 02:06:57 smithi159 systemd[1]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.i.service: Consumed 4min 29.726s CPU time. 2024-03-19T02:06:57.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:06:57.711 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-19T02:06:57.711 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-19T02:06:57.711 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.l 2024-03-19T02:06:57.955 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:06:57 smithi159 systemd[1]: Stopping Ceph mds.l for d1e857f4-e591-11ee-95c9-87774f69a715... 2024-03-19T02:06:58.455 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:06:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l[97804]: 2024-03-19T02:06:58.043+0000 7f4c06ad0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-19T02:06:58.455 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:06:58 smithi159 ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l[97804]: 2024-03-19T02:06:58.043+0000 7f4c06ad0700 -1 mds.l *** got signal Terminated *** 2024-03-19T02:07:01.111 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:07:00 smithi159 podman[104649]: 2024-03-19 02:07:00.789971391 +0000 UTC m=+2.863933468 container died 9f5dc9a82c9b118bcb5f0a0cbe84228f0403ee7601ea58ff775965d30539f97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-19T02:07:01.394 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:07:01 smithi159 podman[104649]: 2024-03-19 02:07:01.202325371 +0000 UTC m=+3.276287443 container cleanup 9f5dc9a82c9b118bcb5f0a0cbe84228f0403ee7601ea58ff775965d30539f97c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c851f2a8983bc273fa71dd1a3e84be4b397f9c2099e180242b453d87e7c2b639, name=ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240318.181317-b6ae249, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-03-19T02:07:01.394 INFO:journalctl@ceph.mds.l.smithi159.stdout:Mar 19 02:07:01 smithi159 bash[104649]: ceph-d1e857f4-e591-11ee-95c9-87774f69a715-mds-l 2024-03-19T02:07:01.614 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d1e857f4-e591-11ee-95c9-87774f69a715@mds.l.service' 2024-03-19T02:07:01.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-19T02:07:01.651 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-19T02:07:01.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d1e857f4-e591-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-19T02:07:01.848 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T02:07:04.285 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d1e857f4-e591-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-19T02:07:04.476 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T02:07:07.073 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d1e857f4-e591-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-19T02:07:07.263 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T02:07:09.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T02:07:09.767 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T02:07:09.801 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-19T02:07:09.830 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-19T02:07:09.833 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/remote/smithi092/crash 2024-03-19T02:07:09.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/crash -- . 2024-03-19T02:07:09.872 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-19T02:07:09.872 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-03-19T02:07:09.874 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/remote/smithi144/crash 2024-03-19T02:07:09.875 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/crash -- . 2024-03-19T02:07:09.934 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-19T02:07:09.934 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-03-19T02:07:09.937 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/remote/smithi159/crash 2024-03-19T02:07:09.938 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/crash -- . 2024-03-19T02:07:09.978 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-19T02:07:09.978 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2024-03-19T02:07:09.979 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-19T02:07:09.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-19T02:07:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:2024-03-19T01:44:40.854917+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T02:07:10.013 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-19T02:07:10.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[SEC\]' /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-19T02:07:10.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]' /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-19T02:07:10.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[WRN\]' /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-19T02:07:10.231 INFO:teuthology.orchestra.run.smithi092.stdout:2024-03-19T01:44:40.854917+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-19T02:07:10.231 INFO:tasks.cephadm:Compressing logs... 2024-03-19T02:07:10.231 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-19T02:07:10.273 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-19T02:07:10.276 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-19T02:07:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-03-19T02:07:10.298 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-03-19T02:07:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-19T02:07:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-19T02:07:10.298 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-19T02:07:10.299 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-19T02:07:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-19T02:07:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-19T02:07:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-19T02:07:10.300 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-19T02:07:10.301 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mgr.x.log: 90.3% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-19T02:07:10.303 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-19T02:07:10.303 INFO:teuthology.orchestra.run.smithi159.stderr:: No such file or directory 2024-03-19T02:07:10.303 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-19T02:07:10.303 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-19T02:07:10.304 INFO:teuthology.orchestra.run.smithi092.stderr: 94.8% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.j.log.gz 2024-03-19T02:07:10.313 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-03-19T02:07:10.313 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-03-19T02:07:10.313 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-19T02:07:10.313 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-19T02:07:10.313 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-19T02:07:10.313 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-19T02:07:10.314 INFO:teuthology.orchestra.run.smithi144.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-19T02:07:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-19T02:07:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.b.log: /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-19T02:07:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-19T02:07:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-19T02:07:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-19T02:07:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.7.log: 90.5% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-19T02:07:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-19T02:07:10.317 INFO:teuthology.orchestra.run.smithi092.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-19T02:07:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-19T02:07:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log 2024-03-19T02:07:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-19T02:07:10.317 INFO:teuthology.orchestra.run.smithi092.stderr: 82.5% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-19T02:07:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log: 87.5% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-19T02:07:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-19T02:07:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-19T02:07:10.318 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-19T02:07:10.318 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.l.log: /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-19T02:07:10.318 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-19T02:07:10.318 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.i.log: /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-19T02:07:10.318 INFO:teuthology.orchestra.run.smithi159.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-19T02:07:10.318 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-19T02:07:10.318 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-19T02:07:10.319 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-19T02:07:10.319 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.f.log: 90.5% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-19T02:07:10.319 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log 2024-03-19T02:07:10.319 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.cephadm.log: 82.1% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-19T02:07:10.319 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-19T02:07:10.319 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log: 87.6% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-19T02:07:10.319 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-19T02:07:10.319 INFO:teuthology.orchestra.run.smithi159.stderr: 92.8% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.2.log: 2024-03-19T02:07:10.319 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-19T02:07:10.320 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-19T02:07:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-19T02:07:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-19T02:07:10.325 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-19T02:07:10.326 INFO:teuthology.orchestra.run.smithi092.stderr: 94.7% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-19T02:07:10.327 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mgr.y.log: 92.8% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-19T02:07:10.340 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log 2024-03-19T02:07:10.341 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.cephadm.log: 82.1% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-19T02:07:10.352 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-19T02:07:10.353 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-19T02:07:10.356 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log: 87.6% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-19T02:07:10.357 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-19T02:07:10.358 INFO:teuthology.orchestra.run.smithi144.stderr: 94.0% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-19T02:07:10.366 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-19T02:07:10.368 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-volume.log: 94.6% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-19T02:07:10.369 INFO:teuthology.orchestra.run.smithi159.stderr: 94.2% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-19T02:07:10.369 INFO:teuthology.orchestra.run.smithi159.stderr: 94.7% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.l.log.gz 2024-03-19T02:07:10.374 INFO:teuthology.orchestra.run.smithi092.stderr: 94.2% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-19T02:07:10.381 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-19T02:07:10.404 INFO:teuthology.orchestra.run.smithi144.stderr: 92.9% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-19T02:07:10.465 INFO:teuthology.orchestra.run.smithi144.stderr: 95.0% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-19T02:07:10.562 INFO:teuthology.orchestra.run.smithi092.stderr: 89.9% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-19T02:07:10.650 INFO:teuthology.orchestra.run.smithi159.stderr: 92.0% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-19T02:07:10.789 INFO:teuthology.orchestra.run.smithi159.stderr: 91.9% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-19T02:07:10.810 INFO:teuthology.orchestra.run.smithi144.stderr: 92.0% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-19T02:07:10.879 INFO:teuthology.orchestra.run.smithi159.stderr: 92.4% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-19T02:07:10.944 INFO:teuthology.orchestra.run.smithi159.stderr: 92.8% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-19T02:07:10.960 INFO:teuthology.orchestra.run.smithi092.stderr: 92.2% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-19T02:07:11.168 INFO:teuthology.orchestra.run.smithi144.stderr: 91.6% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-19T02:07:11.171 INFO:teuthology.orchestra.run.smithi092.stderr: 92.0% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-19T02:07:11.290 INFO:teuthology.orchestra.run.smithi159.stderr: 93.1% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-19T02:07:11.324 INFO:teuthology.orchestra.run.smithi092.stderr: 90.3% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-19T02:07:11.346 INFO:teuthology.orchestra.run.smithi092.stderr: 92.9% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-19T02:07:11.387 INFO:teuthology.orchestra.run.smithi092.stderr: 93.1% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-19T02:07:11.389 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-03-19T02:07:11.389 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m1.104s 2024-03-19T02:07:11.389 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m4.924s 2024-03-19T02:07:11.389 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.181s 2024-03-19T02:07:11.469 INFO:teuthology.orchestra.run.smithi159.stderr: 91.6% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-19T02:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr: 92.8% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-19T02:07:11.673 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-03-19T02:07:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:real 0m1.383s 2024-03-19T02:07:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:user 0m5.440s 2024-03-19T02:07:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m0.205s 2024-03-19T02:07:11.724 INFO:teuthology.orchestra.run.smithi144.stderr: 91.8% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-19T02:07:11.865 INFO:teuthology.orchestra.run.smithi144.stderr: 92.4% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-19T02:07:12.068 INFO:teuthology.orchestra.run.smithi144.stderr: 91.1% -- replaced with /var/log/ceph/d1e857f4-e591-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-19T02:07:12.070 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-03-19T02:07:12.070 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m1.780s 2024-03-19T02:07:12.070 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m6.205s 2024-03-19T02:07:12.070 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m0.238s 2024-03-19T02:07:12.071 INFO:tasks.cephadm:Archiving logs... 2024-03-19T02:07:12.071 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/remote/smithi092/log 2024-03-19T02:07:12.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-19T02:07:12.817 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/remote/smithi144/log 2024-03-19T02:07:12.818 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-19T02:07:13.879 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/remote/smithi159/log 2024-03-19T02:07:13.880 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-19T02:07:14.669 INFO:tasks.cephadm:Removing cluster... 2024-03-19T02:07:14.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d1e857f4-e591-11ee-95c9-87774f69a715 --force 2024-03-19T02:07:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T02:07:15.162 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d1e857f4-e591-11ee-95c9-87774f69a715 --force 2024-03-19T02:07:15.352 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T02:07:15.628 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d1e857f4-e591-11ee-95c9-87774f69a715 --force 2024-03-19T02:07:15.818 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: d1e857f4-e591-11ee-95c9-87774f69a715 2024-03-19T02:07:16.074 INFO:tasks.cephadm:Removing cephadm ... 2024-03-19T02:07:16.074 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-19T02:07:16.092 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-19T02:07:16.106 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-19T02:07:16.121 INFO:tasks.cephadm:Teardown complete 2024-03-19T02:07:16.122 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-19T02:07:16.135 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-19T02:07:16.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-19T02:07:16.138 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-19T02:07:16.148 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-19T02:07:16.264 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-19T02:07:16.274 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-19T02:07:16.275 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T02:07:16.277 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T02:07:16.279 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-19T02:07:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-03-19T02:07:16.291 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-19T02:07:16.293 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-03-19T02:07:16.294 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T02:07:16.294 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-03-19T02:07:16.294 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 57 +1960us[+1980us] +/- 133ms 2024-03-19T02:07:16.294 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -1127us[-1107us] +/- 60ms 2024-03-19T02:07:16.294 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +382us[ +401us] +/- 82ms 2024-03-19T02:07:16.294 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T02:07:16.296 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T02:07:16.296 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-19T02:07:16.296 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 +1989us[+1989us] +/- 133ms 2024-03-19T02:07:16.296 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -1092us[-1096us] +/- 60ms 2024-03-19T02:07:16.296 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +418us[ +414us] +/- 82ms 2024-03-19T02:07:16.296 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T02:07:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-19T02:07:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-03-19T02:07:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +2047us[+2047us] +/- 133ms 2024-03-19T02:07:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -1109us[-1123us] +/- 60ms 2024-03-19T02:07:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 55 +425us[ +411us] +/- 82ms 2024-03-19T02:07:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-19T02:07:16.298 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-19T02:07:16.307 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-19T02:07:16.308 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-19T02:07:16.331 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-19T02:07:16.415 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-19T02:07:16.484 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-19T02:07:16.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T02:07:16.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T02:07:16.586 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T02:07:16.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T02:07:16.615 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-19T02:07:16.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T02:07:16.645 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-19T02:07:16.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-19T02:07:16.681 INFO:teuthology.task.internal:Duration was 3399.271090 seconds 2024-03-19T02:07:16.681 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-19T02:07:16.714 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-19T02:07:16.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-19T02:07:16.716 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-19T02:07:16.719 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-19T02:07:16.758 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T02:07:16.759 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T02:07:16.761 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-19T02:07:17.127 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-19T02:07:17.127 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-03-19T02:07:17.127 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-19T02:07:17.147 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-03-19T02:07:17.147 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-19T02:07:17.190 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-03-19T02:07:17.191 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-19T02:07:17.221 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-19T02:07:17.221 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-19T02:07:17.224 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-19T02:07:17.233 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-19T02:07:17.289 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-19T02:07:17.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-19T02:07:17.292 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-19T02:07:17.301 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-19T02:07:17.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-19T02:07:17.677 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-19T02:07:17.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-19T02:07:17.704 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-19T02:07:17.731 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-19T02:07:17.759 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-19T02:07:17.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-19T02:07:17.772 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-19T02:07:17.774 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-19T02:07:17.795 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-03-19T02:07:17.797 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-03-19T02:07:17.824 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-03-19T02:07:17.847 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-19T02:07:17.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T02:07:17.865 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-19T02:07:17.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T02:07:17.900 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-19T02:07:17.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-19T02:07:17.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-19T02:07:17.926 INFO:teuthology.task.internal:Transferring archived files... 2024-03-19T02:07:17.927 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/remote/smithi092 2024-03-19T02:07:17.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-19T02:07:17.970 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/remote/smithi144 2024-03-19T02:07:17.971 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-19T02:07:18.014 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283/remote/smithi159 2024-03-19T02:07:18.015 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-19T02:07:18.053 INFO:teuthology.task.internal:Removing archive directory... 2024-03-19T02:07:18.054 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-19T02:07:18.056 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-19T02:07:18.059 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-19T02:07:18.102 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-19T02:07:18.115 INFO:teuthology.task.internal:Not uploading archives. 2024-03-19T02:07:18.115 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-19T02:07:18.140 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-19T02:07:18.141 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-19T02:07:18.143 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-19T02:07:18.145 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-19T02:07:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: 265960 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 19 02:07 /home/ubuntu/cephtest 2024-03-19T02:07:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 19 02:07 /home/ubuntu/cephtest 2024-03-19T02:07:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 19 02:07 /home/ubuntu/cephtest 2024-03-19T02:07:18.160 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-19T02:07:18.174 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-19T02:07:18.243 INFO:teuthology.nuke:Checking targets against current locks 2024-03-19T02:07:18.267 DEBUG:teuthology.nuke:shortname: smithi092 2024-03-19T02:07:18.267 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T02:07:18.292 DEBUG:teuthology.nuke:shortname: smithi144 2024-03-19T02:07:18.292 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T02:07:18.296 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:48:25.049898', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDodgWjm5KBLAUCgPEqL6PcfPJyqSgTP63zjcD/2kOYd58LAmNW5zN0DW4oxUxJ0vGTakCrym7Oza9JneQ8Qas4='} 2024-03-19T02:07:18.311 DEBUG:teuthology.nuke:shortname: smithi159 2024-03-19T02:07:18.311 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-19T02:07:18.314 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:48:25.051219', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPcgf3bF3xOYK0jSmjAbx92BWc8p/GwxO36T7l69Ur4v9h1TBC3+I3HAUT7Z1FTLwwl6VZM7dkTMl0XLGOuUM3A='} 2024-03-19T02:07:18.341 INFO:teuthology.orchestra.console.smithi092:Power off 2024-03-19T02:07:18.341 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-19T02:07:18.354 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609283', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-19 00:48:25.048381', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFytUTgr0V6wZPcYc/2NVkHEvKQSgGFiplTT0K+kTA9nc0hnOjnedq4YECj44Zu3+yZ4rqw5sJdsu32/5StP8TA='} 2024-03-19T02:07:18.363 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2024-03-19T02:07:18.363 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:18.384 INFO:teuthology.orchestra.console.smithi144:Power off 2024-03-19T02:07:18.384 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-19T02:07:18.396 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-19T02:07:18.407 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-03-19T02:07:18.407 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:18.421 INFO:teuthology.orchestra.console.smithi159:Power off 2024-03-19T02:07:18.421 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-19T02:07:18.431 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-19T02:07:18.443 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-03-19T02:07:18.444 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:18.465 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-03-19T02:07:22.397 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:22.432 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:22.466 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:22.508 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-19T02:07:22.542 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-19T02:07:22.576 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-03-19T02:07:26.510 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:26.544 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:26.577 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:26.620 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-19T02:07:26.653 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-03-19T02:07:26.687 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-03-19T02:07:26.755 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-03-19T02:07:26.788 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-03-19T02:07:30.625 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-19T02:07:30.735 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-03-19T02:07:30.836 INFO:teuthology.orchestra.console.smithi092:Power off completed 2024-03-19T02:07:30.941 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} duration: 3399.2710902690887 failure_reason: '"2024-03-19T01:44:40.854917+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-19T02:07:30.942 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-19T02:07:31.035 INFO:teuthology.run:FAIL